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Executive  Summary 


This  report  details  the  demonstration  of  statistical  tools  developed  by  Pacific  Northwest 
National  Laboratory  (PNNL)  and  Sandia  National  Laboratory  to  identify  preferred 
geophysical  transect  designs  for  sites  suspected  of  containing  UXO  and  to  analyze  the 
resulting  geophysical  anomaly  data.  The  analysis  tools  developed  provide  the  Visual 
Sample  Plan  (VSP)  user  the  ability  to  identify  potential  target  areas,  delineate  boundaries 
of  the  target  areas  wherein  UXO  is  most  likely,  and  estimate  the  anomaly  density  within 
each  identified  and  delineated  target  area.  The  methods  are  also  useful  for  providing 
justification  for  no  further  investigation  in  very  large  portions  of  investigation  sites. 

The  PNNL/Sandia  characterization  design  team  received  information  on  a  simulated 
munitions  site  from  Mitretek  Systems  where  4.2”  mortars,  155  mm  munitions,  and 
differing  size  precision  bombs  were  used.  Mitretek  provided  varying  degrees  of 
information  for  each  impact  area  in  the  conceptual  site  model  and  the  characterization 
team  used  the  available  information  to  develop  geophysical  transect  survey  designs  to 
identify  potential  targets  areas  located  in  each  impact  area.  The  resulting  anomaly  data 
from  the  transect  designs  were  analyzed  and  potential  target  areas  were  identified  and 
delineated.  Once  the  potential  target  areas  were  identified,  delineated  and  anomaly 
densities  estimated,  the  truth  data  was  provided  by  Mitretek  and  an  evaluation  of  the 
performance  of  the  statistical  methods  was  conducted.  This  report  summarizes  the 
performance  of  the  statistical  tools  in  identifying  and  delineating  target  areas  and 
estimating  anomaly  density. 

The  simulated  site  consisted  of  5  major  areas  (ARA-1,  ARA-2A,  ARA-2B,  SRA-1,  and 
BGR-1)  ranging  from  6200  acres  to  74650  acres  in  size.  Three  of  the  areas  were  used  as 
artillery  ranges,  one  site  was  used  as  a  small  arms  practice  range,  and  the  one  was  a 
precision  bombing  range.  Multiple  transect  sampling  approaches  were  requested  for 
each  site  by  the  PNNL/SNL  team  to  adequately  evaluate  the  performance  of  VSP  and  the 
geostatistical  methods  for  transect  design,  target  identification  and  delineation,  and 
anomaly  density  estimation.  Using  the  most  conservative  approach  for  each  of  the  5 
areas,  a  summary  of  the  target  detection  and  UXO  containment  with  the  target  area 
boundaries  is  shown  in  Table  EL 


Impact 

Area 

T  ransect 
Spacing 

Density  of 
Target 
Areas 
above 
background 

Percent 
of  Site 
Covered 
with 

Transects 

Total 

Number 

of 

Target 

Areas 

Total 
Number 
of  UXO 

Number 
of  Target 
Areas 
Identified 

Number 
of  UXO 
Contained 

Percent 
of  Target 
Areas 
Identified 

Percent  of 
UXO 

Contained 

ARA-2B 

282m 

>6 

0.71% 

9 

253 

9 

253 

100.00% 

100.00% 

ARA-2A 

344m 

>6 

0.60% 

3 

175 

3 

175 

100.00% 

100.00% 

SRA-1 

204m  X  204m 

>6 

1 .94% 

None 

None 

None 

None 

Correct 

Correct 

ARA-1 

344m 

>6 

0.66% 

7 

110 

7 

110 

100.00% 

100.00% 

BGR-1 

483m  X  961m 

>6 

1 .20% 

24 

92049 

24 

92049 

100.00% 

100.00% 

ARA-2B 

282m 

<6 

0.71% 

8 

19 

1 

0 

12.50% 

0.00% 

ARA-2A 

344m 

<6 

0.60% 

2 

38 

1 

27 

50.00% 

71 .05% 

SRA-1 

204m  X  204m 

>6 

1 .94% 

None 

None 

None 

None 

Correct 

Correct 

ARA-1 

344m 

<6 

0.66% 

5 

73 

1 

31 

20.00% 

42.47% 

BGR-1 

483m  X  961m 

<6 

1 .20% 

No  target  areas  with  density  less  than  6  per  acre. 

Table  El:  Target  identification  and  UXO  containment  performance  of  PNNL/Sandia  for 
the  conservative  designs  separated  by  target  density 


Major  findings  and  conclusions  of  this  demonstration  are  summarized  below. 

•  100%  of  the  target  areas  where  the  anomaly  density  was  >6  anomalies  per  acre 
above  background  were  identified  for  all  areas  where  target  areas  existed. 

•  100%  of  the  UXO  was  contained  within  the  derived  target  area  boundaries  for  all 
areas  where  the  target  area  anomaly  density  was  >6  /acre  above  background. 

•  When  no  target  areas  existed  on  a  suspected  impact  area  (SRA-1),  none  were 
identified. 

•  For  areas  where  target  area  anomaly  density  was  very  close  to  background  density 
(<6/acre),  success  of  identifying  those  target  areas  was  mixed  (not  a  surprise 
given  our  transect  design  criteria). 

•  Transect  width  appears  to  significantly  affect  the  ability  to  identify  target  areas, 
especially  those  with  densities  closer  to  background.  The  2  m  wide  transects 
performed  better  than  1  m  wide  transects. 

•  Target  areas  were  detected  and  delineated  with  very  little  transect  area  coverage 
which  ranged  from  0.6%  to  1.94%  . 

•  Anomaly  density  estimates  for  the  non-BGR  target  areas  were  very  accurate 
(7.6%  average  accuracy).  For  the  BGR  target  areas  the  average  accuracy  was 
20%. 

•  Simulated  target  area  sizes  were  not  always  consistent  with  the  assumed  sizes  and 
shapes  used  to  design  transects.  A  unified  approach  to  this  should  be  pursued. 

•  Some  inconsistencies  in  the  simulated  anomalies  affected  the  performance  of  the 
detection  and  estimation  methods  (see  Appendix  B  for  detailed  description  of  this 
issue). 
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The  PNNL/SNL  combined  approach  resulted  in  a  very  efficient  and  consistent  overall 
approach  to  transect  design,  target  identification  and  delineation,  and  anomaly  density 
estimation. 


1.  Introduction 


1.1  Background 

The  Department  of  Defense  (DoD)  is  in  the  process  of  assessing  and  remediating  closed, 
transferred,  and  transferring  (CTT)  ranges.  Many  of  these  sites  involve  very  large 
geographical  areas  such  that  it  is  often  impractical  and/or  cost  prohibitive  to  perform 
100%  surveys  across  the  entire  site  of  interest.  With  over  20  million  acres  of  land  where 
UXO  may  possibly  reside,  effective  characterization  and  detection  technologies  are 
needed.  Statistical  sampling  methods  and  tools  are  needed  to  support  several  phases  of 
the  characterization  and  decision  making  process  including  optimal  sampling  designs  for 
target  detection,  target  delineation,  anomaly/UXO  density  estimation,  and  post¬ 
remediation  verification. 

Statistical  tools  have  been  developed  under  SERDP/ESTCP  to  support  detection, 
delineation,  density  estimation,  and  mapping  of  unknown  target  areas  where  UXO  are 
most  likely.  Through  ESTCP  sponsorship,  these  tools,  developed  by  Pacific  Northwest 
National  Laboratory  (PNNL)  and  Sandia  National  Laboratory  (SNL),  are  being 
demonstrated  on  simulated  sites  and  on  actual  DoD  sites  suspected  of  UXO  presence. 

The  tools  are  being  deployed  via  the  Visual  Sample  Plan  (VSP)  software. 

Previously,  three  simulated  sites  were  studied  and  the  performance  of  PNNL/SNL  tools 
was  evaluated.  For  each  site,  the  PNNL/SNL  team  was  provided  with  a  site  map  and 
some  information  relative  to  past  site  practices.  Based  on  required  detection  probability 
rates  and  likely  fragmentation  dispersion  patterns,  the  optimal  transect  spacing  was 
identified  and  a  selected  set  of  geophysical  transects  were  requested  and  simulated.  An 
independent  contractor  used  their  site  simulation  software  and  provided  PNNL  and  SNL 
with  all  locations  along  the  requested  transects  where  anomalies  were  detected.  PNNL 
and  SNL  then  applied  their  target  area  detection,  delineation,  and  mapping  tools.  The 
three  site  evaluations  were  performed  sequentially  with  increasing  site  complexity  added 
as  tests  progressed.  This  final  report  pertains  to  the  fourth  and  final  simulated  test  site 
performance  evaluation. 

1.2  Objectives  of  the  Demonstration  and  Site  Description 

This  demonstration  is  the  fourth  and  final  demonstration  of  the  methods  and  tools  on  a 
simulated  site.  The  objective  was  to  exercise  the  tools  in  such  a  way  as  to  ensure  their 
applicability  on  a  variety  of  real  sites,  given  the  unknowns  that  may  be  encountered  at 
these  sites.  The  site  scenario  was  chosen  to  simulate  reality  and  the  difficulties  that  will 
normally  present  themselves  in  the  practical  application  of  statistically-based 
characterization  tools.  This  final  demonstration  site  is  considerably  larger  than  any  of  the 
previous  three  simulated  sites  and  contains  five  distinct  firing  and  bombing  ranges. 

The  following  scenario  is  assumed  for  this  demonstration:  A  large  site  exists,  portions  of 
which  were  used  as  firing  ranges  and  other  portions  were  used  as  bombing  ranges. 
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Boundaries  around  the  general  impact  areas  are  known  and  all  target  areas  were  within 
the  suspected  impact  areas.  Nearly  all  target  area  locations  were  unknown  with  only  one 
exception  where  prior  information  on  the  general  location  of  the  target  area  was  provided. 
A  100%  geophysical  survey  is  not  feasible  due  to  cost,  scale  of  the  site,  and  accessibility 
issues. 

It  is  necessary  to  derive  a  geophysical  transect  sampling  scheme  and  apply  post-survey 
analysis  methods  to  1)  Detect  all  target  areas  of  concern  and  2)  Delineate  (establish  a 
boundary  around)  each  target  area  or  cluster  of  target  areas,  and  3)  Estimate  the  anomaly 
density  for  each  target  area  or  target  area  cluster. 

To  ensure  that  we  fully  tested  out  the  statistical  methods  and  tools,  PNNL/Sandia 
requested  that  the  following  simulated  site  characteristics  be  included  in  the  simulated 
site  and  were  assumed  to  be  components  of  the  site  conceptual  model  (CSM) : 

•  Multiple  munitions  used;  mortars,  projectiles,  bombs. 

•  Multiple  target  areas;  some  clustered;  some  isolated;  some  singleton  targets 

•  Some  areas  of  the  site  with  different  background  anomaly  density  (gradually 
varying;  not  very  large  differences) 

•  Varying  ordnance  dispersion  pattern  models 

•  An  area  with  unusually  high  anomaly  density  but  no  OE  related  anomalies. 

•  Prior  information  on  the  general  location  of  some  of  the  target  areas  as  derived 
from  an  archived  search  report  or  conceptual  site  model. 

The  site  boundaries  and  some  information  regarding  site  history,  munitions  used  and 
suspected  impact  or  target  areas  were  provided  to  PNNL  and  SNL.  After  PNNL/SNL 
designed  and  requested  transects,  the  site  simulators  provided  a  list  of  anomaly  locations 
identified  from  the  simulated  geophysical  surveys  of  the  requested  transect  designs. 
PNNL  and  SNL  then  applied  their  tools  to  identify  suspected  target  areas  and,  in  some 
cases,  requested  additional  transects.  The  site  simulators  updated  the  list  of  anomaly 
locations  to  include  those  found  within  the  additional  transects.  Finally,  the  developed 
tools  were  applied  to  identify  target  areas,  delineate  the  target  areas,  estimate  anomaly 
densities,  and  develop  anomaly  density  maps  of  the  site.  Performance  was  assessed 
based  on  the  criteria  outlined  in  Sections  3  and  4  below. 

1.3  Regulatory  and  Stakeholder  Issues 

At  the  onset  of  this  demonstration,  a  meeting  was  held  with  the  Technical  Advisory 
Committee  (TAC)  consisting  of  State  and  EPA  regulators  as  well  as  Army  Corps  of 
Engineers  personnel.  High  level  objectives  for  this  demonstration  were  discussed  along 
with  our  recommendations  for  various  scenarios  that  would  help  test  our  methods.  The 
TAC  provided  specific  guidance  to  the  ESTCP  program  manager  and  to  those  responsible 
for  simulating  the  sites  and  has  been  cognizant  of  this  particular  demonstration.  Also,  in 
an  effort  to  support  regulator  acceptance  and  technology  transfer,  a  presentation  was 
provided  in  Las  Vegas  to  an  ad  hoc  EP A/COE  committee  set  up  to  deal  with  UXO  sites. 
This  presentation  included  preliminary  findings  on  this  simulated  demonstration.  Efforts 
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to  integrate  the  VSP  UXO  module  with  other  software  and  methods  for  determining 
target  area  size  and  shape  characteristic  of  various  munitions’  usage  were  discussed  along 
with  additional  VSP  enhancements. 

2.  Technology  Description 

2.1  Technology  Development  and  Application 

To  advance  the  state-of  the-art  in  statistical  methods  and  tools  to  meet  its  specific  needs, 
DoD  invested  in  three  SERDP  projects  in  FYOl  at  PNNL,  Sandia  and  ORNL.  The 
projects  responded  to  DoD’s  call  for  “Statistical  Sampling  for  Unexploded  Ordnance 
(UXO)  Site  Characterization  (CUSON-01-01).  Because  each  phase  of  characterization 
supports  a  different  objective  with  different  decision  criteria,  it  quickly  became  apparent 
that  different  statistical  methods  would  be  required  for  each  phase  of  the  characterization 
/  remediation  /  verification  process.  DoD  continued  to  fund  research  at  PNNL  and 
Sandia  to  develop  statistical  methods  that  addressed  different  but  related  phases  of  the 
process. 


PNNL’s  methods  address  SERDP’s  statement  of  need  for  developing  statistical  sampling 
methods  where  100%  surveys  are  unattainable  or  cost  prohibitive.  The  innovation  of 
these  developments  in  target  detection  statistical  sampling  design  methodology  allows  the 
stakeholders  to  explicitly  balance  the  risk  of  not  finding  areas  of  potential  UXO 
contamination  when  they  may  be  present  and  the  cost  of  unnecessary  or  non-optimal 
searches.  These  sampling  design  methods  are  consistent  with  systematic  planning 
processes  such  as  the  Data  Quality  Objectives  (DQO)  process.  The  target  detection 
design  methods  focus  on  determining  the  number  and  spacing  of  transects  that  will  be 
required  to  have  an  acceptably  high  probability  of  traversing  and  detecting  target  areas  of 
a  critical  size,  shape,  and  anomaly  density.  Target  areas  are  defined  as  areas  where  UXO 
are  most  probable  and  the  number  of  geophysical  anomalies  is  much  higher  than  in  the 
background. 


There  are  several  design  parameters  (DQO  parameters)  that  affect  the  probability  of 
detecting  a  target  zone  of  interest  and  the  feasibility  of  the  sampling  design.  These 
include: 

2  2 

•  The  assumed  shape  (circular  or  elliptical)  and  size  (area  in  m  or  ft  )  of  a  target 
area  that  must  be  detected  with  high  probability. 

•  The  required  probability  that  at  least  one  transect  traverses  some  portion  of  the 
assumed  target  area. 

•  The  swath  width  or  footprint  of  the  geophysical  sensor  used  to  detect  anomalies. 

•  The  transect  pattern  (parallel,  square  grid,  or  rectangular  grid  patterns  can  be 
assumed). 

•  The  spatial  orientation  (N-S,  E-W)  of  the  transect  pattern  with  respect  to  expected 
orientation  of  the  target  area. 
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•  The  anomaly  density  patterns  [A  uniform  pattern  (unchanging  density  over  the 
target  area)  or  a  bivariate  normal  pattern  (highest  density  at  the  center  of  the  target 
area  with  decreasing  density  in  all  directions)  are  assumed], 

•  The  critical  anomaly  density  threshold  value  above  which  a  potential  target  area 
will  be  flagged. 

•  The  false  negative  detection  error  rate  of  the  geophysical  sensor  instrument  to  be 
used. 

•  The  costs  for  transect  and  geophysical  sensor  setup  and  sampling/evaluation  costs 
per  linear  foot  of  transect. 

The  effect  of  varying  each  of  these  parameters  can  be  evaluated  and  the  most  feasible, 
cost-effective  sampling  design  can  be  determined. 

2.1.1  Visual  Sample  Plan  (VSP)  Software  Implementation  of  Target  Detection 
Sampling 


PNNL  has  incorporated  its  developments  in  target  detection  statistical  sampling  design 
methodology  into  a  sampling  design  software  program,  Visual  Sample  Plan  (VSP),  which 
has  been  developed  for  soil  and  surface  environmental  characterization  efforts.  The  U.S. 
Department  of  Energy  (DOE),  the  U.S.  Environmental  Protection  Agency  (EPA),  and  the 
Department  of  Defense  (DoD)  are  co-sponsoring  development  of  the  software.  VSP 
allows  the  user  to  import  or  draw  a  map  of  the  facility  site  and/or  area  to  be  characterized. 
Figure  1  shows  a  screen  shot  from  the  VSP  UXO  Target  Detection  sampling  module 
depicting  a  hypothetical  site  with  the  area  where  transect  sampling  is  planned.  The  two 
windows  in  the  upper  left  and  right  of  Figure  2  illustrate  the  VSP  dialog  box  that  allows 
the  user  to  input  the  DQO  parameters  listed  above  and  other  relevant  design  parameters. 
VSP  calculates  the  required  transect  spacing  and  the  probability  of  both  traversing  and 
detecting  a  target  area  of  the  specified  size,  shape,  and  anomaly  density.  The  lower 
figure  in  Figure  2  shows  the  recommended  transects  generated  by  VSP  for  the  selected 
design. 


Given  the  design  parameters,  VSP  will  compute  the  required  spacing  of  transects  to 
achieve  the  specified  probability  of  traversing  the  critical  target  area,  calculate  the 
probability  of  both  traversing  and  detecting  the  critical  target  zone  if  it  exists,  display  the 
proposed  transects  on  the  site  map  and  will  output  the  x,y  coordinates  of  the  proposed 
transects.  The  user  can  then  conduct  a  sensitivity  analysis  by  evaluating  the  effects  of  the 
input  parameters  and  their  required  DQO  parameters.  These  methods  and  tools  allow  the 
project  team  to  balance  DQO  objectives  against  costs  and  other  site  constraints.  After 
analysis  of  the  transect  sampling  data,  if  no  target  zones  are  identified  and  estimated 
anomaly  densities  as  calculated  with  geostatistical  mapping  (see  below)  are  acceptably 
low,  then  plans  for  no  further  action  are  considered.  If  potential  target  zones  are 
identified  through  sampling,  then  the  next  phase  of  sampling  focuses  on  defining  the 
boundary  of  the  target  zone  and  developing  a  sufficiently  accurate  map  of  the 
anomaly/UXO  density. 
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2.1.2  VSP  Target  Identification  flagging  routines 


VSP’s  target  identification  algorithm  uses  a  circular  window  to  calculate  the  density  of 
anomalies  within  a  certain  distance  from  the  center  of  the  window.  This  window, 
centered  on  a  transect,  systematically  moves  along  each  transect  of  the  site  and  flags  the 
center  point  of  the  window  located  on  the  transect  as  an  area  of  high  density  when  the 
window  has  more  anomalies  than  expected  for  background  anomalies  alone.  Because 
there  is  often  no  prior  estimate  of  background  anomalies,  VSP  provides  the  capability  of 
examining  the  distribution  of  densities  based  on  the  user  defined  window  diameter.  The 
user  can  then  determine  an  optimum  critical  value  or  background  density  based  on  this 
window  size.  With  the  optimum  window  size  and  appropriate  background  density 
determined,  potential  target  areas  are  identified.  Figure  3  is  a  screen  shot  of  a  sample 
area  with  the  results  from  the  flagging  routine  (square  boxes)  based  on  the  anomalies 
located  on  the  surveyed  transects.  The  flagged  areas  represent  locations  of  high  anomaly 
density  which  could  be  identified  as  target  areas. 
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Figure  1 :  VSP  Screen  Shot  Showing  Site  Map  and  Area  To  Be  Surveyed 
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Figure  2:  Transect  design  options  and  output  in  VSP.  The  upper  left  figure  represent  the  target 
area  assumptions  tab  and  the  upper  right  figure  represent  the  transect  design  assumptions  tab. 
The  lower  figure  depicts  the  placed  transects  in  the  identified  sample  area. 
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Figure  3:  VSP  Screen  Shot  Showing  Flagged  Areas  of  High  Density 


2.1.3  Geostatistics  Applied  to  UXO  Issues 

Sandia’s  research  extends  geostatistical  algorithms  designed  for  the  analysis  and 
modeling  of  spatially  correlated  data  to  applications  in  the  UXO  site  characterization 
arena.  The  basic  idea  in  the  application  of  geostatistics  to  UXO  applications  is  that  for 
any  site  there  will  only  be  limited  transect  sampling  and  it  is  therefore  necessary  to 
estimate  the  presence  or  absence  of  UXO  at  unsampled  locations  and  to  identify  the 
uncertainty  in  those  estimates.  Under  SERDP  funding,  Sandia  worked  on  four  separate 
tasks:  1)  development  of  a  theoretical  basis  for  the  mapping  of  UXO  contaminated  areas 
as  a  doubly- stochastic  Poisson  process;  2)  development  of  a  sampling  protocol  for  initial 
sampling  transects  and  follow-up  sampling  transects  for  the  practical  field 
characterization  of  UXO  sites;  3)  implementation  of  a  data  worth  approach  for 
determining  the  optimal  location  for  follow  up  sampling  transects  (straight  or 
meandering)  and  the  necessary  number  of  transects  to  meet  site  characterization  goals; 
and  4)  validation  of  the  procedures  against  a  series  of  exhaustively  known  hypothetical 
and  actual  field  data  sets.  The  work  reported  here  extends  this  fourth  task. 

Sandia  developed  a  general  approach  for  integrating  prior  information  with  transect  data. 
One  means  of  generating  prior  information  was  to  use  a  Poisson  process  simulator  to 
allow  for  numerical  representation  of  site  conceptual  models.  This  simulator  allows  the 
user  to  include  uncertainty  inherent  in  the  location  and  sizes  of  targets  and  firing  zones  as 
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determined  from  archival  information  and  existing  geophysical  sampling.  Techniques  for 
the  optimal  placement  of  transects  based  on  this  type  of  site  conceptual  model  have  also 
been  investigated.  Cokriging  has  been  evaluated  as  an  updating  algorithm  to  combine  the 
site  conceptual  model  (prior  information)  with  geophysical  sampling  along  transects  to 
produce  a  map  showing  the  probability  of  being  within  a  target  area  at  all  locations.  This 
probability  map  can  be  compared  against  a  design  reliability  to  make  decisions  (add  to 
dig  list  or  leave  as  is)  for  all  locations  across  the  site.  Initial  validation  on  actual  and 
simulated  sites  shows  that  cokriging  provides  accurate  representations  of  the  number  of 
UXO  even  in  the  presence  of  spatially  biased  prior  information.  In  addition  to  mapping 
the  probability  of  being  within  a  target  area,  techniques  have  also  been  developed  for  the 
direct  mapping  of  geophysical  anomaly  density  from  transect  data. 

2.1.4  Probability  Mapping 


For  sites  with  some  existing  data,  it  may  be  advantageous  to  create  maps  showing  the 
probability  of  exceeding  a  specified  intensity  of  geophysical  anomalies  per  location  or  the 
probability  of  having  at  least  one  UXO,  or  one  anomaly  of  interest,  per  location.  These 
measures  can  roughly  be  qualified  as  mapping  the  probability  of  being  within  a  target 
area  at  any  location.  Typically,  areas  with  higher  probability  of  high  anomaly  density  or 
UXO  occurrence  are  target  areas  within  a  site.  These  types  of  maps  are  especially  useful 
for  locating  additional  sampling  transects  and  allowing  the  regulator  to  define  the 
reliability  of  the  characterization  decisions  made  from  the  available  data.  Probability 
values  near  0.50  indicate  areas  of  maximum  uncertainty  with  respect  to  the  threshold 
being  considered.  As  an  example.  Figure  4,  A  shows  the  locations  of  four  sampling 
transects  taken  on  a  hypothetical  site  created  with  a  doubly  stochastic  Poisson  simulator. 
For  each  location  along  the  transect  the  probability  of  finding  at  least  one  UXO  is  set  to 
1.0  (UXO  is  present)  or  0.0  (no  UXO  are  present).  In  many  applications  where 
geophysical  surveys  are  completed  prior  to  any  excavation,  the  probability  map  may  be 
created  showing  the  probability  of  an  anomaly  of  interest  existing,  or  not  existing,  at  any 
location.  An  anomaly  of  interest  is  an  anomaly  that  exceeds  a  prescribed  geophysical 
threshold  value  or  meets  a  certain  fitness  value  when  evaluated  against  the  simulated 
geophysical  response  for  different  munitions  believed  to  exist  at  the  site. 


Probability  mapping  can  be  completed  using  only  the  information  obtained  from  the 
transects  as  designed  using  VSP  and  the  appropriate  design  parameters  (see  above),  or  the 
geostatistical  algorithms  can  be  used  to  merge  both  the  transect  data  and  more  subjective 
information  on  the  location  and  extent  of  target  areas  as  obtained  from  Archival  Search 
Reports  (ASR’s),  aerial  imagery  and/or  simulations  of  UXO  locations  based  on  the 
conceptual  site  model  (CSM).  For  the  example  calculation  shown  in  Figure  4,  prior 
information  in  the  form  of  probabilities  on  the  location  of  two  targets  and  a  firing  zone  as 
derived  from  the  CSM  is  shown  in  Figure  4,  B. 


Figures  4,  C  and  D  show  the  results  of  estimating  the  probability  of  at  least  one  UXO 
across  the  site.  Estimates  are  only  made  up  to  a  distance  away  from  the  nearest  sample 
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point  that  is  equal  to  the  range  of  spatial  correlation  as  defined  by  the  sample  data 
obtained  in  the  transects.  Figure  4,  C  shows  the  estimates  made  using  only  the  transect 
sample  data.  Figure  4,  D  integrates  the  conceptual  site  model  information  (Figure  4,  B) 
with  the  sample  transect  data  by  using  the  geostatistical  estimator  as  a  Bayesian  operator. 
Note  the  change  in  size  and  shape  of  the  areas  of  near  0.50  probability  (green  areas)  when 
the  estimates  are  constrained  to  the  conceptual  site  model  information.  Areas  with 
probability  values  near  0.50  are  areas  of  greatest  uncertainty  and  correspond  to  transition 
zones  between  targets  and  background.  Focused  sampling  in  these  areas  can  efficiently 
determine  the  boundaries  of  the  target  areas. 

2.1.5  Density  Mapping 


Similar  to  the  probability  mapping,  estimates  of  geophysical  anomaly  density  can  be 
made  across  the  site  based  on  limited  data.  The  same  geostatistical  estimation  algorithms 
(kriging  and  cokriging)  used  for  probability  mapping  can  incorporate  the  site-specific 
calibration  of  the  geophysical  instrument  into  the  density  mapping  of  the  site.  Density 
mapping  can  be  used  with  limited  or  even  larger  amounts  of  data  to  accurately  estimate 
the  geophysical  anomaly  density  in  smaller  areas  of  the  site  prior  to  developing  a  final 
detailed  survey  map  or  dig  list.  Density  estimation  requires  that  some  finite  area  of  the 
site  be  defined  and  then  the  number  of  anomalies  per  area  is  estimated.  Typically,  we 
have  used  a  regular  grid  where  each  cell  in  the  grid  is  an  equal  area  and  the  number  of 
anomalies  per  grid  cell  is  estimated.  Grid  cell  areas  have  ranged  from  625  to  over 
5000m  .  Even  though  the  density  estimates  are  often  made  on  grid  cells  with  areas  much 
less  than  one  acre,  the  results  are  generally  reported  in  units  of  anomalies  per  acre  as  this 
seems  to  be  the  standard  unit  of  measure  in  the  UXO  community. 

Density  estimation  also  allows  for  target  area  boundary  delineation.  A  density  contour  is 
selected  and  areas  of  the  site  where  the  estimated  density  exceeds  this  contour  are  defined 
as  being  within  the  target  area.  While  this  target  area  delineation  approach  is  more 
straightforward  than  the  probability  mapping  approach,  it  is  a  deterministic  approach  and 
does  not  allow  the  decision  maker  any  direct  consideration  of  reliability  in  setting  the 
extent  of  the  target  boundary. 
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A 


B 


H^othetical  Site  Sample  Transects  Gaussian  Prior  Probability 


Figure  4:  Locations  along  sample  transects  (A)  are  coded  as  1.0  (at  least  one  UXO 
exists)  or  0.0  (no  UXO  exists).  The  conceptual  site  model  (B)  provides  estimates  of  the 
probability  of  having  at  least  one  UXO  at  any  location  based  on  surface  surveys  and 
historical  records.  The  probability  of  at  least  one  UXO  existing  is  then  mapped  to  all 
locations  that  are  within  the  spatial  correlation  range  of  a  transect  sample  using  just  the 
transect  data  (C)  and  integrating  the  conceptual  site  model  with  the  transect  data  (D). 
The  color  scale  shows  the  probability  of  at  least  one  UXO  at  every  location. 
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2.2  Previous  Testing  of  the  Technology 

Three  tests  were  conducted  previously  on  simulated  sites  with  increased  complexity 
added  for  each  of  the  tests.  In  each  case,  some  initial  site  information  was  provided  to 
both  the  PNNL  and  SNL  teams,  transects  were  requested  and  anomaly  data  from  the 
transects  were  provided  by  Mitretek.  In  all  three  cases,  the  PNNL  and  SNL  teams 
worked  independently  and  results  are  reported  for  each  team.  A  summary  of  the  results 
of  each  of  these  tests  is  provided  Table  1 .  A  complete  review  of  these  sites  and  the 
results  can  be  read  in  the  demonstration  plan  for  this  study. 

In  each  of  the  three  scenarios  PNNL  identified  all  the  target  areas,  contained  100%  of  the 
UXO  in  each  of  the  identified  target  areas,  and  minimized  the  amount  of  area  selected 
outside  of  the  actual  target  areas.  PNNL  also  demonstrated  that  the  VSP  user  can  choose 
from  parallel,  square  grid,  or  rectangular  grid  transect  spacing  to  apply  in  different 
regions  of  the  sample  area. 


Scenario 

%  of  Targets 
Identified 

%  of  the  UXO 
Identified 

%  of  the  sample  area  covered  by 
transects 

Simulation  1 

100% 

100% 

0.77% 

Simulation  2 

100% 

100% 

0.84% 

Simulation  3 

100% 

100% 

0.26% 

Table  1:  Previous  Performance  of  PNNL’s  independent  analysis  of  the  first  three 

simulated  sites. 


The  Sandia  approach  to  identifying  transect  locations  was  an  adaptive  one  with  prior 
information  on  the  site  dictating  where  the  initial  transects  would  be  located.  For  each 
site  these  initial  transects  were  obtained  and  then  additional  transects  were  requested.  At 
each  site,  the  probability  of  the  anomaly  density  exceeding  a  threshold  value  was  mapped 
onto  a  relatively  fine  grid  using  the  geostatistical  algorithms.  Then  a  probability  contour 
was  selected  and  all  cells  in  the  map  above  that  contour  level  were  identified  as  being 
part  of  the  target.  The  results  are  given  in  terms  of  the  design  reliability  which  is  the 
complement  of  the  probability  of  being  within  a  target.  For  example,  the  map  showing 
the  probability  of  being  within  a  target  is  created,  a  probability  contour  level  of  0.05  is 
selected  and  all  locations  with  a  5  percent  or  greater  chance  of  being  within  a  target  are 
classified  as  the  target  region.  The  5  percent  probability  contour  serves  as  the  boundary 
of  the  target.  The  design  reliability  is  the  complement  of  the  probability  threshold  or  95 
percent  in  this  example. 

For  each  of  the  previous  simulated  sites,  the  percent  of  the  target  areas  found,  the  percent 
of  the  UXO  within  the  target  boundaries  and  the  percent  of  the  site  incorrectly  identified 
as  target  areas  (false  positives)  are  recorded.  For  each  one  of  these  performance 
measures,  the  design  reliability  necessary  to  produce  these  results  is  also  identified.  The 
performance  measures  and  the  design  reliabilities  are  presented  in  Table  2.  In  general, 
the  higher  the  design  reliability  the  more  conservative  the  decision  and  the  greater  the 
number  of  targets  and  UXO  that  are  identified.  The  flip  side  of  this  relationship  is  that 
more  false  positives  are  created  as  the  decision  becomes  more  conservative.  Results  in 
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Table  2  show  a  range  of  design  reliabilities  that  produced  the  same  results  in  terms  of 
identifying  targets  and  UXO.  False  positive  rates  were  kept  to  20  percent  or  less  at  alls 
sites. 


Mitretek 

Site 

Targets 

Found 

(%) 

Design 

Reliability 

UXO  in 
target 
bounds  (%) 

Design 

ReliabiUty 

False 

Positives 

(%) 

Design 

Reliability 

1 

100 

>=  0.50 

100 

>=  0.50 

20 

<=  0.80 

2 

100 

>=  0.85 

93 

>=  0.85 

5 

<=  0.85 

3 

80 

>=  0.90 

94 

>=  0.80 

10 

<=  0.90 

Table  2:  Summary  of  Sandia  results  for  the  three  previous  simulated  sites. 

3.  Demonstration  Design 


3.1  Performance  Objectives 

This  fourth  simulated  site  demonstration  introduced  more  challenging  site  conditions  and 
scenarios  to  evaluate  the  effectiveness  of  the  combined  tools  of  PNNL  and  Sandia. 
Contrary  to  the  previous  three  simulated  site  demonstrations,  this  demonstration  took  an 
integrated  approach  between  PNNL  and  Sandia  as  shown  in  the  following  flow-diagram 
(Figure  5).  Specific  performance  objectives  for  this  study  are  listed  in  Table  3. 
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Int^rated  Methods/Tools  Approach 


Figure  5:  Integrated  PNNL/Sandia  Methods/Tool  Approach. 
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Type  of 
Performance 
Objective 

Primary  Performance  Criteria 

Expected  Performance 

Quantitative 

%  of  UXO  encompassed  with 
suspected  bounded  areas 

100%  unless  in 
completely  inaccessible 
areas 

%  of  Target  areas  encompassed  within 
suspected  bounded  areas 

100%  unless  in 
completely  inaccessible 
areas 

Accurate  distinction  between  target 
areas  and  high  density  non-OE  related 
areas. 

No  non-target  areas 
flagged  as  target  areas 

%  of  land  area  covered  by  transects 

<10% 

Accurate  estimate  of  anomaly  density 
for  each  suspected  target  area 

Within  20% 

Qualitative 

Minimal  amount  of  area  outside  of 
true  target  areas  included  in  bounded 
suspected  target  areas. 

Minimize 

Ability  to  appropriately  handle 
inaccessible  areas  and  inexact 
transects  (transects  that  are  not 
perfectly  parallel) 

General  Use 
Demonstration 

Table  3:  Performance  objectives  for  PNNL/Sandia  team 


3.2  Test  Site 

This  simulated  site  contains  five  different  areas  with  a  combined  area  of  over  300  sq.  km 
as  designed  by  Mitretek.  Mitretek  provided  a  conceptual  site  model  (CSM)  scenario  for 
each  of  the  five  impact  areas  and  the  CSM  suggested  that  target  areas  existed  in  each  of 
these  impact  areas.  However,  the  information  about  target  areas  varies  from  approximate 
location  to  no  knowledge  of  a  location.  The  rest  of  the  site  has  no  known  indications  of 
target  areas. 

3.3  Testing  and  Evaluation  Plan  (DQO  Approach) 

The  approach  for  testing  and  evaluating  the  performance  of  these  tools  is  outlined  below 
by  taking  a  Data  Quality  Objectives  (DQO)  approach  to  the  problem. 

3.3.1  Problem  Statement 

A  large  site  exists,  portions  of  which  were  used  as  firing  ranges  and  other  portions  were 
used  as  bombing  ranges  and  munitions  target  areas  and  all  target  areas  were  within  the 
suspected  impact  areas.  Nearly  all  target  area  locations  are  unknown  with  one  exception 
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where  the  general  location  was  provided.  A  100%  geophysical  survey  is  not  feasible  due 
to  cost  and  accessibility  issues. 

It  is  necessary  to  derive  a  geophysical  transect  sampling  scheme  and  apply  post-survey 
analysis  methods  to  1)  Detect  all  target  areas  of  concern  and  2)  Delineate  (establish  a 
boundary  around)  each  target  area  or  cluster  of  target  areas,  3)  Estimate  the  anomaly 
density  for  each  target  area  or  target  area  cluster,  and  4)  Allow  field  analysts  to  determine 
if  UXO  are  indeed  present  in  high  density  areas. 

3.3.2  Identify  the  Decisions 

The  first  objective  of  conducting  a  geophysical  survey  would  be  to  detect  areas  where 
UXO  presence  is  highly  suspect  and  distinguish  these  areas  from  background  areas. 

Once  detected,  the  next  objective  would  be  to  define  a  boundary  that  would  most  likely 
encompass  any  UXO  that  might  be  present.  Once  a  boundary  is  established,  one  would 
attempt  to  estimate  anomaly  density  within  the  bounded  area  and  decide  whether  100% 
geophysical  characterization  would  be  necessary  or  another  less  than  100%  transect 
sampling  design  would  be  most  productive. 

The  key  anomaly  identification  decisions  are  then: 

•  Decision  1  (Detection):  Are  there  indications  that  a  target  area  of  concern  (and 
therefore  suspected  UXO  area)  exists?  (representative  of  suspected  munitions 
used) 

•  Decision  2  (Delineation):  If  so,  what  is  the  boundary  of  the  target  area  and  what 
is  the  average  anomaly  density  estimate  for  the  area? 

3.3.3  Identify  Inputs  to  the  Decisions 

Several  factors  are  considered  for  each  of  the  decisions,  some  of  which  come  from  the 
geophysical  survey  data  and  others  from  stakeholder  input.  The  inputs  to  each  decision 
are  shown  below. 

Inputs  for  Decision  1  (Detection): 

•  Background  anomaly  density 

•  Anomalies  and  anomaly  density  along  transects 

•  Density  Threshold  above  which  indicates  a  target  area  may  exist  (critical  density) 

•  Geophysical  instrument  footprint  and  probability  of  detection 

•  Size  and  shape  of  target  areas  of  concern 

•  Relative  risks  of  making  incorrect  decision. 

•  Prior  knowledge/information  about  target  locations  and  munitions  use 
(Conceptual  Site  Model). 

•  Cost  of  geophysical  surveying. 

•  Areas  where  transects  cannot  be  obtained 

•  Design  reliability  value  at  which  the  target  boundary  should  be  defined. 
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Inputs  for  Decision  2  (Delineation): 

•  Background  density  estimate 

•  Anomalies  and  anomaly  density  within  a  unit  area 

•  Density  threshold  above  which  indicates  a  target  area  may  exist  (critical  density) 

•  Risk  of  too  narrowly  defining  the  boundary  vs.  setting  the  boundary  too  wide. 

•  Anomaly  locations  from  any  additional  transects  within  the  suspected  areas. 

•  Cost  of  geophysical  surveying. 

•  Segment  of  transect  over  which  average  density  is  calculated. 

•  Spatial  correlation  model  of  the  sampled  intensity  and/or  indicator  data 

•  Cost  per  anomaly  for  removal  and/or  detailed  surveying 

3.3.4  Define  the  Boundaries  of  the  Study 

The  site  contains  five  different  areas  with  a  combined  area  of  over  300  sq.  km  and  fixed 
by  the  site  simulation  contractor.  The  conceptual  site  model  defines  a  different  scenario 
for  each  of  the  five  impact  areas.  The  CSM  suggests  that  known  targets  existed  in  these 
impact  areas  within  the  site,  although  exact  locations  of  all  targets  are  not  known.  The 
rest  of  the  site  has  no  known  indications  of  target  areas. 

This  study  only  addresses  the  target  detection  and  delineation  and  does  not  address 
further/no  further  survey  questions,  discrimination,  identification,  or  prioritization  of 
anomalies.  Known  target  and  UXO  locations  were  not  revealed  to  PNNL/Sandia  team 
until  after  the  first  two  decisions  above  were  made.  It  is  assumed  that  the  simulated  site 
is  representative  of  an  actual  site  and  dispersion  patterns  for  each  munition  are 
representative  of  actual  patterns. 

Decisions  1  and  2  above  will  be  applied  separately  to  each  suspected  target  area  or  target 
area  cluster  in  each  impact  area. 

3.3.5  Develop  Decision  Rules 

Decision  rules  are  in  the  form  of  if. .  .then. .  .else  statements.  The  following  decision  rules 
and  variations  will  be  employed. 

Decision  1  (Detection):  If  the  anomaly  density  within  an  area  of  size  xxx  is  estimated  to 
be  greater  than  yyy  then  that  area  is  identified  as  a  suspect  target  area  of  interest,  (xxx 
and  yyy  would  be  determined  through  analysis  of  the  anomaly  data) 

Decision  2  (Delineation):  If  an  area  is  identified  as  a  suspect  target  area  of  interest  and  it 
is  contiguous  with  other  identified  suspect  target  areas,  those  areas  will  be  combined  into 
a  single  bounded  area  and  the  boundary  will  completely  enclose  that  area.  If  an  area  is 
identified  as  a  suspect  target  area  of  interest  but  is  isolated,  then  either  a  boundary  will  be 
developed  that  completely  encloses  the  area,  or  additional  transects  will  be  requested  to 
further  explore  the  suspected  target  area  and  refine  it’s  boundaries,  or  based  on  an 
evaluation  of  the  surrounding  anomaly  patterns  and  the  sensitivity  to  changes  in  the 
decision  threshold,  yyy,  it  may  be  removed  from  the  suspect  target  area  list. 
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If  the  probability  of  exceeding  the  density  threshold  is  greater  than  www  then  a  contour 
boundary  will  be  drawn  to  include  those  areas  where  the  probability  of  exceeding  the 
threshold  exceeds  www. 

The  values  of  xxx,  yyy,  www,  and  zzz  will  be  varied  for  these  analyses  to  evaluate  and 
demonstrate  their  effect  on  performance. 

3.3.6  Specify  Tolerable  Limits  on  Decision  Errors 

There  are  two  types  of  decision  errors  for  each  of  the  decisions  as  shown  below. 

Decision  1  Errors  (Detection): 

•  Type  I:  Conclude  that  there  is  no  target  area  of  concern  within  an  area  when  in 
fact  there  is  UXO  present  in  that  area  (False  Negative  error). 

•  Type  II:  Conclude  that  an  area  is  a  target  area  of  concern  when  in  fact  there  is  no 
UXO  present  (False  Positive  error). 

These  error  types  are  generalized  to  include  conclusions  of  too  few  (Type  I)  or  too  many 
(Type  II)  target  areas  as  they  may  be  multiple  target  areas  within  a  survey  area. 

The  probability  of  making  a  type  I  decision  error  is  dependent  on  the  transect  spacing,  the 
size  and  shape  of  OE  related  fragment  distribution,  and  the  geophysical  survey 
instrument  detection  rate.  The  probability  of  making  a  type  II  decision  error  could  be 
affected  by  varying  background  anomaly  densities,  the  difference  between  the 
background  anomaly  density  and  the  anomaly  density  within  a  target  area.  For  purposes 
of  this  demonstration,  the  error  rates  were  controlled/evaluated  by 

•  Ensuring  at  least  a  aaa%  chance  of  traversing  and  detecting  an  elliptical  target 
area  ofbbb  meters  in  width  and  ccc  meters  in  length  of  un/known  orientation  that 
may  be  representative  of  the  size  and  shape  of  a  target  area  with  fragment 
dispersion  for  various  munitions. 

For  this  demonstration,  different  values  ofbbb,  and  ccc  and  how  they  affect  the  decision 
error  rates  were  evaluated. 

Decision  2  Errors  (Delineation): 

•  Type  I:  The  target  boundary  does  not  include  all  UXO  associated  with  a 
particular  target  area  (False  Negative  error).  The  probability  of  having  a  certain 
number  of  anomalies  outside  any  defined  target  area  will  be  quantified. 

•  Type  II:  The  target  boundary  includes  a  much  larger  area  than  necessary,  thereby 
increasing  unnecessary  characterization,  digging,  and  cost  (False  Positive  error). 

The  delineation  decision  is  driven  by  the  estimated  anomaly  density.  A  density  decision 
threshold  is  selected  and  any  areas  above  that  threshold  are  enclosed  within  the  boundary. 
For  this  demonstration,  the  methodology  used  and  the  effect  of  selecting  various  decision 
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thresholds  will  be  examined.  The  type  I  error  was  eontrolled  to  have  no  more  than  a  5% 
chance  of  a  UXO  falling  outside  the  delineated  areas  (design  reliability  equals  95%).  The 
type  II  error  will  be  explored  by  comparing  the  acreage  of  the  delineated  areas  for  various 
decision  thresholds  used. 

3.3.7  Optimize  the  Design 

These  tools  optimize  the  transect  design  by  providing  the  analyst  with  a  balance  between 
cost  and  identification.  This  demonstration  shows  how  VSP  tools  can  be  used  to 
maximize  information  from  the  minimum  of  transect  surveys  while  enabling  the 
researcher  to  know  the  power  of  the  transect  design  and  their  level  of  certainty  in  the 
decision  making  process.  Where  power  is  defined  as  l-[3  and  [3  is  the  probability  of  a 
type  II  error  as  defined  in  the  Decision  1  Errors  (Detection)  section  above. 

4.  Performance  Assessment 

4.1  Performance  Criteria 

Assumptions  that  affect  the  transect  sampling  scheme  (i.e.,  fragment  distribution  patterns, 
required  traversing/detecting  probabilities,  and  prior  knowledge)  were  varied  and 
multiple  transect  options  were  requested  by  PNNL/Sandia  for  each  impact  area.  This 
provided  an  evaluation  of  the  sensitivity  of  the  results  to  assumptions  and  showed  the 
tools  available  in  VSP.  Table  4  lists  the  performance  criteria  assessed  in  this  report. 


Performance 

Criteria 

Description 

Primary  or 
Secondary 

%  UXO  Within 
Boundary 

The  %  of  UXO  that  fall  within  the  boundaries  of 
the  identified  target  areas. 

Primary 

%  UXO  Within 
Boundary  if  Target 

Area  has  a  Sufficient 
Density 

The  %  of  UXO  that  fall  within  the  boundaries  of 
the  identified  target  areas  with  a  tme  target  area 
density  of  greater  than  6  anomalies  per  acre  above 
background. 

Primary 

%  of  Target  Areas 
Identified 

The  %  of  all  tme  target  area  locations  that  were 
identified  and  encompassed  within  the 
boundaries. 

Primary 

%  of  Target  Areas 
Identified  if  Target 

Area  has  a  Sufficient 
Density 

The  %  of  all  tme  target  area  locations  with  a 
target  area  density  greater  than  6  anomalies  per 
acre  above  background  that  were  identified  and 
encompassed  within  the  boundaries 

Primary 

Target  Area 
Identification 

The  accurate  distinction  between  target  areas  and 
high  density  non-OE  related  areas. 

Secondary 

Land  Area  Covered 

%  of  land  area  traversed  by  geophysical  transects. 
Should  be  small  relative  to  entire  land  area. 

Primary 

Accurate  Anomaly 
Density  Estimation 

Accurate  estimate  of  anomaly  density  and  number 
of  anomalies  within  each  suspected  target  area. 

Primary 

Unnecessary  Inclusion 
of  Land  Mass  Inside 
Bounded  Areas. 

Boundaries  around  the  suspected  target  areas  are 
sufficiently  accurate  to  minimize  the  future  dig 
area  to  include  the  UXO. 

Secondary 

Table  4:  Performance  criteria  for  simulation  demonstration 
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4.2  Performance  Confirmation  Methods 


PNNL  and  Sandia  evaluated  the  site  characteristics  and  historical  information  to 
recommend  several  sets  of  transect  designs.  Mitretek  processed  the  transects  and 
provided  the  anomaly  locations  found  along  the  requested  transects.  The  statistical 
analysis  tools  developed  by  PNNL  and  Sandia  were  used  to  identify  suspected  target 
areas.  Additional  transects  were  requested  to  better  define  the  anomaly  characteristics 
within  some  of  the  suspected  impact  areas.  Mitretek  supplied  an  additional  list  of 
anomaly  locations  based  on  the  additional  transect  requests  and  the  estimates  for  the 
target  area  locations  and  boundary  delineations  were  finalized.  Estimates  of  the 
number/density  of  anomalies  in  each  target  area  and  boundaries  of  the  target  areas  were 
derived  and  provided  to  Mitretek.  Finally,  Mitretek  provided  the  actual  simulated  data 
showing  the  location  of  all  targets  and  UXO  and  the  simulated  background  levels  for 
each  area.  An  evaluation  of  the  performance  of  the  statistical  methods  per  the  criteria 
outlined  above  is  described  in  the  following  sections. 

4.2.1  Process  of  Transect  Design  and  Analysis 

The  process  of  target  area  identification  can  follow  a  four  stage  process  -  Design, 
Detection,  Iteration,  and  Delineation.  Each  stage  is  important  for  developing  the  correct 
conclusions  about  the  potential  target  areas  in  wide  area  assessment.  The  original 
demonstration  plan  proposed  a  fifth  step  of  dig  identification.  This  step  was  removed 
from  demonstration  by  ESTCP  and  Mitretek  did  not  provide  data  in  a  manner  that  this 
step  could  be  performed. 

A  proper  transect  design  must  be  created  to  attain  the  desired  results.  At  this  stage  all 
prior  information  about  the  site  in  an  archive  search  report  (ASR)  or  conceptual  site 
model  (CSM)  is  used  to  decide  on  the  different  parameters  and  assumptions  necessary  for 
the  transect  design.  Figure  2  shows  the  tabs  within  the  VSP  software  that  allow  the  user 
to  input  the  required  parameters  and  assumptions.  A  target  area  size  and  shape  are 
defined  based  on  the  munition  used  in  the  site  and  the  dispersion  pattern  of  the  used 
munitions,  as  shown  in  Figure  8.  The  user  defines  the  transect  width,  the  orientation  of 
the  target  to  transect  design,  and  chooses  a  transect  pattern — square,  parallel,  or 
rectangular.  The  transect  spacing  is  dependent  upon  target  area  assumptions  and  the 
required  probability  of  traversing  a  target  area. 

Target  Identification  begins  after  the  data  have  been  gathered  from  the  proposed  transect 
designs.  The  target  identification  algorithm,  currently  in  VSP,  implements  a  circular 
window  to  calculate  the  density  of  anomalies  within  a  certain  distance  from  the  center  of 
the  window.  This  window,  centered  on  a  transect,  systematically  moves  along  each 
transect  of  the  site  and  flags  an  area  as  high  density  when  the  defined  moving  window 
has  more  anomalies  than  expected  for  background  anomalies  alone.  Because  there  is 
often  no  prior  estimate  of  background  anomalies,  VSP  provides  the  capability  of 
examining  the  distribution  of  densities  based  on  a  user  defined  window  diameter.  The 
user  can  then  determine  an  optimum  critical  value  or  background  density  based  on  this 
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window  size.  With  the  optimum  window  size  and  appropriate  baekground  density 
determined,  potential  target  areas  are  identified.  Figures  6  and  7  show  the  VSP  tabs  that 
perform  the  target  area  identifieation  funetions  in  VSP.  At  this  stage  additional  transects 
may  be  requested  to  better  identify  potential  target  areas.  This  process  is  iterative  and 
additional  transect  requests  could  occur  depending  on  the  uncertainties  of  the  flagged 
areas  and  the  time  and  finances  allocated  to  the  remediation.  After  all  of  the  desired 
anomaly  data  based  on  the  requested  transects  are  obtained,  a  final  target  area  detection 
routine  is  run  and  all  potential  target  areas  are  identified. 

Finally,  the  boundaries  of  the  identified  (flagged)  target  areas  are  delineated.  VSP  will 
provide  two  methods  to  identify  the  perimeter  of  potential  target  areas.  Currently  VSP 
allows  the  user  to  visualize  the  size  of  the  flagged  target  areas  and  then  manually  create  a 
boundary  (created  as  a  convex  hull  of  straight-line  segments)  that  surrounds  the  flagged 
areas  of  concern.  VSP  is  in  the  final  stages  of  integrating  SNL’s  geostatistical 
methodology  to  estimate  the  density  and  perimeter  of  a  target  area.  The  geostatistical 
approach  supports  two  different  means  of  delineating  the  target  boundary:  1)  a  map  is 
created  defining  the  probability  of  being  within  a  target  area  across  the  entire  site. 
Mapping  the  probability  allows  the  decision  makers  to  select  the  probability  with  which 
they  are  most  comfortable.  For  example,  if  all  locations  with  a  5  percent  or  greater 
chance  of  being  within  a  target  area  are  delineated  as  target  areas  (95  percent  reliability) 
then  any  location  outside  of  this  boundary  has  a  less  than  5  percent  chance  of  actually 
being  a  target  area  -  or  the  Type  I  error  is  controlled  to  be  less  than  5  percent  at  any 
location.  2)  a  map  of  anomaly  density  is  created  for  the  entire  site  and  locations  where 
the  estimated  density  exceeds  the  critical  density  are  delineated  as  targets.  This  second 
approach  does  not  allow  for  a  direct  control  of  decision  errors,  but  provides  a  more 
intuitive  understanding  of  the  target  boundary  as  it  is  based  directly  on  anomaly  density 
estimates. 
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Figure  6.  VSP  Dialog  Box  On  Target  Area 
Detection 
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Figure  7.  VSP  graph  which  shows  the 
distribution  of  anomaly  densities  based  on  a 
defined  window  size. 
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Figure  8:  Diagram  of  Derivation  of  Target  Area  Size/Shape  of  Concern 
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4.3  Data  Analysis,  Interpretation  and  Evaluation 

Mitretek  provided  SimRangE  Scenario  4,  which  contained  the  5  impact  areas  shown  in 
Figure  9.  It  was  assumed  that  4.2”  mortars  and  155mm  projectiles  were  the  main 
munitions  used  on  the  site.  One  larger  impact  area  was  used  for  aerial  bomb  training  of 
live  and  smoke  filled  bombs  (M30  1001b,  M31  3001b,  M43  5001b,  M34  20001b  bombs) 
and  one  smaller  area  was  used  for  small  arms  training.  PNNL/Sandia  developed  multiple 
transect  designs  for  4  of  the  5  impact  areas  with  ARA-1  only  having  one  transect  design 
created.  The  different  transect  designs  for  the  same  impact  areas  were  meant  to  provide 
an  opportunity  to  check  sensitivity  of  decision  results  to  target  area  assumptions. 


Figure  9:  Complete  SimRangeE  Scenario  with  the  five  simulated  impact  areas 

This  simulation  contained  some  targets  areas  that  had  very  low  anomaly  densities.  In 
ARA-2B  there  were  5  targets  that  had  a  density  above  background  of  less  than  1  anomaly 
per  acre.  A  total  of  15  of  the  58  targets  had  a  density  above  background  of  less  than  6 
anomalies  per  acre.  Tables  5  and  6  summarize  by  scenario  the  number  of  targets  that  had 
a  density  less  than  6  anomalies  per  acre  above  background  or  greater  than  6  anomalies 
per  acre  above  background  respectively. 
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Impact 

Area 

T  ransect 
Spacing 

Percent  of 
Site 

Covered 

with 

Transects 

Total 
Number 
of  Target 
Areas 

Total 
Number 
of  UXO 

ARA-2B 

345m 

0.58% 

8 

19 

ARA-2B 

282m 

0.71% 

8 

19 

ARA-2A 

344m 

0.60% 

2 

38 

ARA-2A 

298m 

0.33% 

2 

38 

ARA-1 

344m 

0.66% 

5 

73 

BGR-1 

483x961m 

1 .20% 

0 

0 

BGR-1 

232m 

0.86% 

0 

0 

Table  5:  Summary  of  simulation  for  each  scenario  based  on  targets  areas  with  densities  less 
than  6  anomalies  per  acre  above  background 


Impact 

Area 

Transect 

Spacing 

Percent 
of  Site 
Covered 
with 

Transects 

Total 
Number 
of  Target 
Areas 

Total 
Number 
of  UXO 

ARA-2B 

345m 

0.58% 

9 

253 

ARA-2B 

282m 

0.71% 

9 

253 

ARA-2A 

344m 

0.60% 

3 

175 

ARA-2A 

298m 

0.33% 

3 

175 

ARA-1 

344m 

0.66% 

7 

110 

BGR-1 

483x961m 

1 .20% 

24 

92049 

BGR-1 

232m 

0.86% 

24 

92049 

Table  6:  Summary  of  simulation  for  each  scenario  based  on  target  areas  with  a  density 
above  background  of  6  anomalies  per  acre 

4.3.1  Artillery  Range  Area  1  (ARA-1) 

Artillery  Range  Area  1  covered  2 1 ,000  acres  and  had  primary  targets  towards  the  center 
of  the  range.  Each  of  the  targets  was  fired  upon  from  multiple  staging  sites  by  4.2” 
mortars  and  155mm  projectiles.  The  PNNL/Sandia  team  was  also  provided  the  location 
and  area  of  “observed”  terrain  scaring.  The  observed  terrain  scaring  was  located  at  the 
center  of  the  ARA-1  site  and  had  a  diameter  of  approximately  2  km.  As  discussed  below, 
this  information  was  used  to  investigate  the  incorporation  of  prior  information  into 
geostatistical  estimation  process. 

4.3.1. 1  ARA-1  Transect  Design 

One  transect  design  was  developed  for  this  impact  area  based  on  the  potential  427  by  280 
m  target  area  size  of  4.2”  mortars,  as  shown  in  Figure  8.  Three  north-south  transects 
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were  requested,  with  the  original  east-west  transects,  that  would  have  a  high  likelihood  of 
traversing  the  potential  central  target  areas.  The  2  m  wide  east-west  transects,  shown  in 
Figure  10,  were  344  m  apart,  covered  0.66%  of  the  ARA-1  impact  area  and  had  a  95% 
probability  of  traversing  the  target  area  of  interest.  Based  on  the  results  from  the 
preliminary  target  identification  no  additional  transects  were  requested  to  better 
discriminate  potential  areas. 


gl'  Vliwl  5^4  Wyi  -  1fi  an# 

gn*  im  infPiriMh  iMfc  fimi*  kwi  I*-  ^  - 

-  -q  /  a  ^ 


I 

Figure  10:  ARA-1  transect  design  with  identified  anomalies 

4.3.1.2  ARA-1  Target  Area  Identification  Results 

Two  potential  target  areas  were  identified  in  this  impact  area.  These  target  areas  were 
derived  using  both  the  perimeter  estimates  from  the  VSP  flagging  routine  and  SNL’s 
geostatistical  perimeter  estimation.  Figure  1 1  shows  the  final  perimeter  estimates  for 
ARA-1  created  with  the  VSP  flagging  routine  along  with  the  actual  maximum  size  target 
areas  based  on  the  simulation  results  from  Mitretek  and  Figure  12  shows  the  perimeter 
estimates  with  the  UXO  plotted  on  the  target  areas.  The  actual  simulated  targets  did  have 
some  overlap  which  would  increase  the  density  per  acre  in  the  overlapping  area.  Except 
for  one  target,  all  low  density  target  areas  were  the  farthest  away  from  the  other  target 
areas.  Each  target  area  is  color  coded  by  its  density  above  background  (18.21  anomalies 
per  acre)  as  calculated  by  dividing  the  total  number  of  simulated  fragments  and  UXO  by 
the  maximum  perimeter  of  each  simulated  target  area.  For  any  target  with  a  density  of  6 
anomalies  per  acre  or  higher  above  background,  the  target  area  identification  tools 
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contained  100%  of  the  UXO.  This  site  contained  5  target  areas  with  an  anomaly  density 
similar  to  background  and  the  lowest  density  area  did  not  contain  any  UXO.  For  the 
overall  site  77%  of  the  UXO  was  contained  within  the  PNNL/Sandia  delineated  target 
areas.  These  results  are  summarized  in  Table  7. 


Density  of 
Target  Areas 
above 
background 

Transect 

Spacing 

>6 

344m 

344m 

<6 

Percent  of 
Site 

Covered 

with 

Transects 

0.66% 

0.66% 


Total 

Number 

of 

Target 

Areas 

Total 
Number 
of  UXO 

Number 
of  Target 
Areas 
Identified 

Number  of 
UXO 

Contained 

Percent 
of  Target 
Areas 
Identified 

Percent  of 
UXO 

Contained 

7 

110 

7 

110 

100.00% 

100.00% 

5 

73 

1 

31 

20.00% 

42.47% 

Table  7:  Target  identification  per: 


'ormance  for  ARA-1. 


Many  combinations  were  examined  between  window  size  and  critical  density.  PNNL 
used  a  900  m  diameter  circular  window  with  a  critical  density  of  34  anomalies  per  acre  as 
the  analysis  values  to  identify  the  finalized  target  areas. 
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Figure  1 1 :  Results  of  ARA-1  with  flagged  areas,  perimeter  delineation  and  target  area 

density  and  size  for  each  target. 
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Figure  12:  UXO  coordinates  plotted  on  ARA-1  map  with  perimeter  delineation,  target 


area  size  and  density  for  each  target. 


Sandia’s  processing  techniques  differed  from  PNNL’s  primarily  in  that  different 
upscaling  techniques  were  used.  SNL’s  approach  was  to  average  the  anomaly  counts 
within  a  rectangular  moving  window  working  within  a  grid-based  framework.  This 
technique  focused  on  a  single  transect  during  the  averaging  process.  For  all  of  the 
simulated  ranges  discussed  in  this  report,  a  rectangular  averaging  window  350  m  in 
length  with  a  width  equal  to  the  transect  sample  width  (typically  2  m)  was  used.  This 
averaging  window  was  centered  at  the  averaging  location  and  oriented  parallel  to  the 
sampling  transect.  These  differences  in  upscaling  techniques  often  lead  to  different 
critical  density  levels  for  target  area  delineation  resulting  from  the  SNL  and  PNNL 
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averaging  techniques.  For  ARA-1  SNL  used  a  critical  density  level  of  65  anomalies  per 
acre  for  target  delineation. 


To  model  the  spatial  variability  of  the  transect  data,  variograms  were  developed  for  each 
of  the  kriging  estimates  presented  here.  Variograms  model  how  the  variance  between 
data  points  changes  as  the  spatial  distance  between  any  two  points  increases.  Typically 
the  variance  increases  as  the  distance  between  points  increases.  Figure  13  presents  the 
indicator  and  standard  variograms  developed  using  the  transect  data  from  ARA-1 .  The 
points  in  this  figure  represent  the  variance  values  computed  at  specific  lag  distances.  The 
solid  lines  represent  the  analytic  models  fit  to  the  data  points.  It  is  these  model  curves 
that  are  used  during  the  kriging  procedures.  The  parameters  describing  these  curves  are 
provided  in  Appendix  A. 

Figure  14  shows  the  indicator  kriging  probability  levels  for  ARA-1  using  an  indicator 
kriging  threshold  of  65  anomalies  per  acre.  Only  areas  with  a  probability  of  0.05  or 
greater  of  being  above  the  65  anomalies  per  acre  threshold  are  indicated  by  color- filled 
contours.  These  areas  were  included  as  part  of  the  information  used  in  determining  the 
final  target  boundaries.  The  two  indicated  areas  falling  outside  of  the  PNNL/SNL  final 
target  boundaries  were  considered  too  small  to  be  actual  targets  based  on  the  available 
range  use  information,  and  so  were  not  included  in  the  final  target  delineation.  This 
decision  was  confirmed  as  proper  after  the  full  simulated  range  data  were  available. 

Figure  15  shows  magnetic  anomaly  density  maps  for  both  the  kriging  estimates  and  the 
original  site  data  for  ARA-1.  The  kriging  estimate  was  created  using  50  m  square  grid 
cell  spacing.  The  actual  anomaly  density  map  was  created  using  all  anomaly  information 
for  the  range  (UXO,  fragments,  background  clutter,  etc.)  as  provided  by  Mitretek  and  was 
generated  by  averaging  the  anomalies  using  the  same  grid  framework  (50  m  cells)  as  used 
in  the  kriging  estimates.  All  density  values  are  presented  as  anomalies  per  acre. 

Comparison  of  the  estimated  and  actual  anomaly  density  maps  shows  that,  using  only 
limited  transect  information,  the  major  features  of  the  magnetic  anomaly  density 
distribution  of  the  simulated  site  are  well  represented  in  the  kriged  estimates.  The 
notable  exceptions  are  the  relatively  small  high-density  area  found  at  the  southern  end  of 
the  simulated  impact  area  which  is  not  represented  in  the  kriged  estimate,  and  several 
sporadic  areas  in  the  kriged  estimates  which  are  above  the  true  density  values.  These 
typically  occur  outside  of  the  simulated  impact  area  within  the  field  of  background 
clutter. 

Figures  16  and  17  show  comparison  box  and  whisker  plots  for  each  of  the  target  areas 
and  for  the  entire  ARA-1  range.  The  data  compared  are  the  grid  cell  averaged  true 
magnetic  anomaly  data  and  the  estimated  kriging  values.  These  figures  show  that  the 
interquartile  range  and  mean  of  the  original  anomaly  data  set  is  well  represented  by  the 
kriged  estimates,  but  the  range  of  the  estimated  values  is  smaller.  This  reduction  in  the 
overall  range  of  values  is  due  to  the  smoothing  nature  of  the  kriging  estimator  and  the 
fact  that  the  extreme  high  and  low  density  values  of  the  true  data  are  not  necessarily 
intersected  by  sampling  transects. 
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Table  8  presents  a  comparison  of  some  basic  statistics  from  the  original  and  estimated 
anomaly  data  sets.  It  shows  that  the  mean  of  the  estimated  density  for  each  of  the 
delineated  target  areas  are  all  within  approximately  15%  of  the  actual  mean,  and  the 
mean  for  the  entire  site  is  within  5%  of  the  actual. 

The  ARA-1  range  was  also  used  as  an  example  of  including  prior  information  into  the 
geostatistical  estimation  techniques.  As  used  here,  prior  information  is  defined  as 
information  independent  of  the  transect  data  which  may  aid  in  target  delineation. 
Typically  this  information  is  anecdotal  or  ‘soft’  in  nature  and  may  be  derived  from 
archival  search  reports  (ASR’s)  and/or  aerial  photography  obtained  during  the  history  of 
the  site  operation. 

As  an  example  of  prior  information,  simulated  range  ARA-1  is  reported  to  have  surface 
scarring  centered  at  XY  location  60,000/50,000,  with  a  diameter  of  approximately  2,000 
meters.  To  test  the  consistency  of  this  prior  information  against  actual  sample  transects,  a 
model  representing  the  prior  information  (Figure  18)  was  developed  and  the  differences 
between  the  model  and  the  transect  data  were  computed.  These  residuals  were  then  used 
to  krig  residual  estimates  for  the  entire  ARA-1  range.  The  kriged  residuals  were  then 
added  to  the  model  of  prior  information  to  obtain  the  final  estimate  of  anomaly  densities. 

The  prior  information  was  modeled  using  a  bivariate  Gaussian  distribution  with  a 
maximum  value  set  equal  to  that  observed  in  the  processed  transect  data  (80.9  anomalies 
per  acre)  and  a  standard  deviation  equal  to  the  range  of  the  variogram  for  the  transect  data 
(900  m).  The  background  density  was  set  to  the  lower  quartile  of  the  transect  data. 

As  a  test  of  including  prior  information  with  only  very  limited  transect  data,  only  six 
transects  were  used  in  the  analysis.  These  were  divided  into  an  equal  number  of  north- 
south  and  east-west  transects  which  intersected  the  area  of  interest  indicated  by  the  prior 
information.  The  locations  of  theses  six  transects  and  the  results  from  kriging  with  only 
these  six  transects  are  shown  in  Figure  19.  Figure  20  shows  the  anomaly  density  estimate 
resulting  from  the  inclusion  of  the  prior  information  along  with  the  six  transects. 
Comparing  Figures  19  and  20  shows  little  difference  between  the  distributions  of 
anomaly  density  estimates.  This  indicates  that  the  prior  information  is  inconsistent  with 
the  field-sampled  transect  data  and  hence  is  likely  incorrect.  This  is  confirmed  by 
examining  the  kriged  estimates  developed  from  using  more  comprehensive  transect  data. 
Figure  21  shows  kriged  estimates  of  magnetic  anomaly  density  developed  using  41 
sample  transects.  As  indicated  by  this  figure,  there  is  no  large,  high-density  feature 
located  at  the  location  surmised  by  the  prior  information.  This  confirms  that  the  original 
prior  information  was  in  someway  biased  or  incorrect. 

To  demonstrate  the  inclusion  of  prior  information  which  is  consistent  with  the  sample 
transect  data,  an  additional  prior  model  was  constructed.  This  model  was  developed  in  a 
similar  manner  as  before  using  the  maximum  observed  densities  along  the  transects  and 
the  anomaly  density  variogram  ranges  as  the  standard  deviation  of  the  bivariate  Gaussian 
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model,  but  for  this  analysis,  the  bivariate  Gaussian  feature  was  eentered  over  high  density 
features  observed  in  the  kriging  estimates  constructed  from  the  comprehensive  (41 
transect)  data.  Two  of  the  high-density  features  observed  in  Figure  21  were  considered  in 
the  development  of  this  example  of  prior  information.  Figure  22  shows  the  model  of 
prior  information  resulting  from  this  process. 

Figure  23  shows  the  result  of  including  prior  information  which  is  consistent  with  the 
transect  data.  The  kriging  estimates  shown  in  this  figure  were  developed  using  the  new 
prior  information  model  and  the  original  six  transects.  As  shown  in  the  figure,  areas  with 
high  anomaly  density  initially  indicated  by  the  transect  data  are  expanded  by  the 
inclusion  of  the  prior  information.  This  expansion  of  the  high  density  areas  results  in  an 
estimated  anomaly  distribution  which  more  closely  resembles  that  developed  from  the  41 
sample  transects. 

The  examples  of  including  prior  information  discussed  above  show  how  this  process  can 
be  used  to  help  establish  the  reliability  of  the  prior  information,  and,  when  the  prior 
information  is  consistent  with  field  observations,  use  the  prior  information  to  help  refine 
estimates  developed  from  transect  data.  A  significant  advantage  of  this  approach  is  that 
the  “hard”  data  coming  from  the  transects  always  overrides  the  “soft”  prior  information, 
and  therefore,  if  the  two  data  sets  are  inconsistent,  the  final  estimate  will  be  more 
consistent  with  the,  presumably,  more  reliable  transect  data. 
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Figure  13:  Variograms  for  indicator  variable  using  a  threshold  of  65  anomalies  per  acre 
(top  plot)  and  anomaly  per  acre  values  (lower  plot)  for  ARA-1.  Points  represent  values 
computed  from  transect  data;  solid  lines  represent  models  fit  to  data  points.  See 
Appendix  A  for  a  listing  of  values  and  explanation  of  the  parameters  used  in  the 
variogram  models. 
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Figure  14:  ARA-1  indicator  kriging  probability  levels  for  an  indicator  threshold  level  of 

65  anomalies  per  acre. 
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Figure  15:  Magnetic  anomaly  density  (anomalies  per  acre)  maps  for  ARA-1.  Upper 
figure  shows  kriged  estimate  with  target  number,  lower  figure  shows  actual  distribution. 
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Figure  16:  Box  and  whisker  plots  comparing  Mitretek  range  anomaly  densities  (Mtk) 
and  kriging  estimated  densities  (KrigEst)  for  individual  target  areas.  Boxes  represent 
interquartile  range  with  median  shown  by  horizontal  line.  Vertical  lines  show  largest 
value  within  upper  limit  (Q3+1.5(Q3-Q1)  and  smallest  value  within  lower  limit  (Ql- 
1.5(Q3-Q1)  with  outliers  beyond  this  range  shown  by  asterisks.  Mean  is  shown  by  circle 
with  cross. 
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Figure  17:  Box  and  whisker  plots  comparing  Mitretek  range  anomaly  densities  (Mtk) 
and  kriging  estimated  densities  (KrigEst)  for  entire  ARAl  range  area.  Boxes  represent 
interquartile  range  with  median  shown  by  horizontal  line.  Vertical  lines  show  largest 
value  within  upper  limit  (Q3+1.5(Q3-Q1)  and  smallest  value  within  lower  limit  (Ql- 
1.5(Q3-Q1)  with  outliers  beyond  this  range  shown  by  asterisks.  Mean  is  shown  by  circle 
with  cross. 


Area 

Source 

Mean 

Mean  Diff. 

Std  Dev 

Min 

Max 

Target  1 

Mtk 

53.4 

-14.8% 

23.3 

13.0 

207.2 

KrigEst 

45.5 

18.6 

6.1 

129.0 

Target  2 

Mtk 

50.8 

-2.8% 

29.3 

11.3 

215.3 

KrigEst 

49.4 

15.4 

15.8 

84.3 

Entire 

Range 

Mtk 

21.6 

-4.6% 

12.2 

1.6 

215.3 

KrigEst 

20.6 

11.5 

0 

129.0 

Table  8:  Comparison  of  estimated  (KrigEst)  and  actual  (Mtk)  magnetic  anomaly  densities 


for  ARA-1  range. 
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Figure  18:  Model  representing 
original  prior  information.  Data 
represent  hypothetical  anomaly 
densities  in  anomalies  per  acre. 


Figure  19:  Magnetic  anomaly 
density  estimates  (anomalies  per 
acre)  for  ARA-1  developed  using 
only  six  transects  and  no  prior 
information. 
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Figure  20:  Magnetic  anomaly 
density  estimates  (anomalies  per 
acre)  for  ARA-1  developed  using 
original  prior  information  model 
and  six  sample  transects. 
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Figure  21:  Magnetic  anomaly 
density  estimates  (anomalies  per 
acre)  for  ARA-1  developed  using  41 
sample  transects. 
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Figure  22:  Model  representing 
revised,  two-target  prior 
information.  Data  represent 
hypothetical  anomaly  densities  in 
anomalies  per  acre. 
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Figure  23:  Magnetic  anomaly 
density  estimates  (anomalies  per 
acre)  for  ARA-1  developed  using 
revised,  two-target  prior  information 
model  and  six  sample  transects. 
Compare  to  Figure  20. 
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4.3.2  Artillery  Range  Area  2A  (ARA-2A) 

Artillery  Range  Area  2  A  is  smaller  than  ARA-1  at  19,000  acres,  but  contained  the  same 
possible  munition  as  ARA-1.  ARA-2A  was  used  less  than  the  other  impact  areas  and  no 
information  about  the  location  or  number  of  target  areas  in  this  impact  area  was  available. 

4.3.2.1  ARA-2A  Transect  Design 

Two  transect  designs  were  developed  for  ARA-2A  based  on  the  potential  427  m  long  by 
280  m  wide  elliptical  target  area.  One  transect  design,  shown  in  Figure  24,  had  2  m 
transects  spaced  344  m  apart  and  the  second  design  (Figure  25)  had  1  m  transects  spaced 
298  m  apart  The  tighter  transect  design  was  based  on  the  assumption  that  lower  use  sites 
could  have  a  smaller  range  and  deflection  probability  for  the  munitions  of  interest. 
Although  the  transect  spacing  was  set  based  on  a  probable  target  area  size  and  a  desired 
probability  of  traversal  (95%),  the  width  of  the  transects  for  each  design  was  varied  to 
examine  detection  sensitivity  to  transect  width.  Based  on  the  preliminary  results 
additional  transects  were  requested  for  the  2m  transect  design  (the  east-west  transects 
shown  in  Figure  24).  Each  area  with  additional  transects  requested  resulted  in  an 
identified  target  area.  The  1  m  transect  design  covered  0.33%  of  ARA-2A  and  the  2  m 
transect  design  covered  0.59%  of  the  impact  area. 


F«r  IW  w  FI  ikmt,  lw«  lir  Ldw  ^  X-4SMH  Jfi 


Figure  24:  Final  requested  transect  design  for  2-m  wide  transects  spaced  344  m  apart  with 

anomaly  locations  for  ARA-2A. 
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Figure  25:  298  m  spaced  1  m  wide  transect  design  for  ARA-2A  with  anomalies 


4.3.2.2  ARA-2A  Target  Area  Identification  Results 

This  impact  area  had  a  background  level  of  10.12  anomalies  per  acre,  5  simulated  target 
areas,  and  1  target  area  based  on  data  from  the  clean  up  at  Ft.  Ritchie.  This  Ft.  Ritchie 
data  was  separated  into  four  different  regions  labeled  as  sectors  in  Figure  26.  These  four 
sectors  (1,  3,  4,  6)  located  next  to  each  other,  should  really  be  analyzed  as  two  different 
target  areas.  Sector  6  was  in  an  open  area  and  only  contained  UXO  --  it  did  not 
contain  any  fragment  locations  from  the  exploded  ordinance.  Sectors  1,3,  and  4 
contained  fragments  and  UXO  but  had  large  un-sampled  areas  due  to  buildings  and 
parking  lots  that  were  not  identified  in  the  ARA-2A  map,  area  summary,  or  as  a  result  of 
the  transect  requests.  Some  of  the  irregularities  associated  with  the  Fort  Ritchie  data  used 
here  are  discussed  further  in  Appendix  B  at  the  end  of  this  report.  Because  of  the 
complexities  of  the  Ft.  Ritchie  data,  these  results  are  reported  separately  from  the 
simulated  target  area  results  in  Table  9.  The  Ft.  Ritchie  results  are  listed  in  Table  10. 


Density  of 
Target 
Areas 
above 
background 

Transect 

Spacing 

Percent  of 
Site 

Covered 

with 

Transects 

Total 

Number  of 
Target 
Areas 

Total 
Number 
of  UXO 

Number 
of  Target 
Areas 
Identified 

Number 
of  UXO 
Contained 

Percent  of 
Target 
Areas 
Identified 

Percent  of 
UXO 

Contained 

>6 

344m 

0.006 

3 

175 

3 

175 

1 

1 

>6 

298m 

0.0033 

3 

175 

3 

175 

1 

1 

<6 

344m 

0.006 

2 

38 

1 

27 

0.5 

0.710526 

<6 

298m 

0.0033 

2 

38 

0 

0 

0 

0 

Table  9:  Target  area  ic 


entification  results  for  ARA-2A  without  Ft.  Ritchie  data 
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Figure  26:  Map  of  Ft.  Ritchie  data  supplied  to  PNNL/Sandia  by  Mitretek. 


Scenario 

Sector 

UXO 

Present 

Fragments 

Present 

Sector 

Size 

(Acres) 

Sector  Area 
Density 
above 
background 

Fragments 

Around 

UXO 

Number 

UXO 

Identified 

Sector 

Identified 

Percent 

UXO 

Identified 

344  m 

1.3,4 

45 

9100 

68 

135 

Y 

45 

Y 

100% 

344  m 

6 

310 

134 

125 

3.54 

N 

304 

Y 

98% 

298  m 

1,3,4 

45 

9100 

68 

135 

Y 

39 

Y 

87% 

298  m 

6 

310 

134 

125 

3.54 

N 

16 

N 

5% 

Table  10:  Ft.  Ritchie  data  separated  by  sectors  with  Fragments  around  UXO 


PNNL  identified  the  final  flagged  target  areas  by  using  a  window  diameter  of  650  m  with 
a  background  density  of  25  anomalies  per  acre  and  a  550  m  diameter  window  with  a 
background  density  of  37  anomalies  per  acre  for  the  344  m  and  298  m  transect  designs 
respectively.  Sandia  identified  their  targets  using  a  rectangular  averaging  window  350  m 
in  length  with  a  width  equal  to  the  transect  sample  width.  This  width  was  1  m  and  2  m 
for  the  298  m  and  344  m  transect  scenarios  respectively.  The  final  critical  density  levels 
determined  by  SNL  for  the  ARA-2A  range  were  46  anomalies  per  acre  for  the  298  m 
transect  spacing,  and  45  anomalies  per  acre  for  the  344  m  spaced  transects. 


In  each  design  all  the  targets  with  target  density  of  6  anomalies  per  acre  or  more  above 
background  were  identified.  The  2  m  wide  transects  that  were  spaced  344  m  apart  did 
identify  one  of  the  targets  that  was  about  5  anomalies  per  acre  above  background.  Figures 
27  and  29  have  the  simulated  values  displayed  with  the  perimeter  estimates  for  the  344  m 
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spaced  transect  design.  Figures  28  and  30  are  similar  graphs  with  the  target  area 
perimeters  for  the  1  m  transects  spaced  298  m  apart. 


Figure  27:  ARA-2A  with  flagging,  final  target  area  perimeters  (black  lines),  and  aetual 
target  areas  eolored  by  density  for  the  344  m  transeet  spacing 


Figure  28:  ARA-2A  with  flagging,  final  target  area  perimeters  (blaek  line),  and  actual 
target  areas  eolored  by  density  for  the  298  m  transeet  spacing 
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Figure  29:  UXO  coordinates  plotted  on  ARA-2A  simulated  target  map  with  perimeter 
delineation  based  (black  lines)  on  the  344  m  spaced  transect  design. 
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Figure  30:  UXO  coordinates  plotted  on  ARA-2A  simulated  target  map  with  perimeter 
delineation  (black  lines)  based  on  the  298  m  spaced  transect  design. 
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Figure  3 1  shows  the  indicator  kriging  probability  levels  for  ARA-2A  using  an  indicator 
kriging  threshold  of  46  and  45  anomalies  per  acre  for  the  298  m  and  344  m  spaced- 
transects  respectively.  Only  areas  with  a  probability  of  0.05  or  greater  of  exceeding  the 
threshold  value  are  shown  by  color-filled  contours.  These  areas  were  included  as  part  of 
the  information  used  in  determining  the  final  target  boundaries.  Variogram  model 
parameters  used  in  the  kriging  estimates  are  listed  in  Appendix  A 

The  Indicator  Kriging  results  for  the  298  m  spaced  transects  (1  m  width)  show  several 
small  isolated  areas  outside  of  the  final  target  delineations.  These  were  considered  too 
small  to  be  actual  target  sites  and  so  were  not  incorporated  into  the  final  target 
boundaries.  These  small  areas  do  not  appear  in  the  344  m  spaced  transect  (2  m  width) 
results.  In  the  I  m  width  transect  scenario  there  were  two  impact  areas  that  were  not 
properly  identified.  Although  one  of  these  appears  as  a  relatively  high-probability  area  in 
the  Indicator  Kriging  results,  its  small  size  precluded  it  from  inclusion  without  additional 
information.  This  same  area  was  identified  using  the  2  m  transects  with  the  addition  of 
supplemental  east-west  transects.  The  single  impact  area  not  identified  in  either  transect 
scenario  appears  to  have  a  relatively  small  footprint  and  low  anomaly  density  (see  Figure 
32). 

Figures  32  and  33  show  the  kriged  estimate  and  actual  cell-averaged  density  for  the  298 
m  and  344  m  spaced  transect  scenarios  respectively.  In  general,  the  kriged  estimates  for 
both  scenarios  provide  a  good  representation  of  the  basic  anomaly  distribution 
considering  the  limited  sampling.  As  shown  by  the  actual  anomaly  density  figures,  the 
one  impact  area  not  identified  in  either  sampling  scenario  does  not  contain  anomaly 
concentrations  significantly  above  the  surrounding  area. 

Figures  34  and  35  show  comparison  box  and  whisker  plots  for  each  of  the  target  areas 
and  for  the  entire  ARA-2A  range.  The  data  compared  are  the  grid  cell  averaged  true 
magnetic  anomaly  data  and  the  estimated  kriging  values  using  data  from  the  1  m  width 
transects.  Similar  information  for  the  2  m  width  transects  are  shown  in  Figures  36 
through  38. 

These  figures  show  that,  in  general,  the  interquartile  range  and  mean  of  the  original 
anomaly  data  set  is  well  represented  by  the  kriged  estimates,  but  the  range  of  the 
estimated  values  is  typically  smaller  than  the  true  data  set  ranges.  This  result  is  typical 
for  estimates  made  with  a  smoothing  algorithm  such  as  kriging. 

Table  1 1  presents  a  comparison  of  some  basic  statistics  from  the  original  and  estimated 
anomaly  data  sets.  It  shows  that  the  mean  of  the  estimated  density  for  each  of  the 
delineated  target  areas  are  all  within  approximately  16%  of  the  actual  mean,  and  the 
mean  for  the  entire  site  is  within  2%  of  the  actual  mean.  With  the  exception  of  Impact 
Area  2,  that  has  some  issues  with  how  the  Fort  Ritchie  data  were  simulated,  the  mean 
density  of  each  target  area  is  within  10  percent  of  the  true  mean  value. 
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Figure  3 1 :  Indicator  kriging  probability  levels  for  ARA-2A.  Top  figure  shows  levels  for 
1  m  wide  transects  spaced  at  298  m;  bottom  figure  shows  levels  for  2  m  wide  transects 
spaced  at  344  m.  Numbers  indicate  final  target  identifiers. 
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Figure  32:  Magnetic  anomaly  densities  in  anomalies  per  acre  for  ARA-2A.  Top  figure 
shows  kriged  density  levels  developed  using  1  m  width  transects;  bottom  figure  shows 

actual  cell-averaged  density  levels. 
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Figure  33:  Magnetic  anomaly  densities  in  anomalies  per  acre  for  ARA-2A.  Top  figure 
shows  kriged  density  levels  developed  using  2  m  width  transects;  bottom  figure  shows 

actual  cell-averaged  density  levels. 
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Figure  34:  Box  and  whisker  plots  for  ARA-2A,  1  m  wide  transeet  seenario  eomparing 
Mitretek  range  anomaly  densities  (Mtekavg)  and  kriging  estimated  densities  (Krigest)  for 
individual  target  areas.  Boxes  represent  interquartile  range  with  median  shown  by 
horizontal  line.  Vertieal  lines  show  largest  value  within  upper  limit  (Q3+1.5(Q3-Q1)  and 
smallest  value  within  lower  limit  (Q1-1.5(Q3-Q1)  with  outliers  beyond  this  range  shown 
by  asterisks.  Mean  is  shown  by  eirele  with  eross. 
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Figure  35:  Box  and  whisker  plots  for  ARA-2A,  1  m  wide  transect  scenario  comparing 
Mitretek  range  anomaly  densities  (Mtekavg)  and  kriging  estimated  densities  (Krigest)  for 
Target  Area  3  (top  plot)  and  entire  site  (bottom  plot).  Boxes  represent  interquartile  range 
with  median  shown  by  horizontal  line.  Vertical  lines  show  largest  value  within  upper 
limit  (Q3+1.5(Q3-Q1)  and  smallest  value  within  lower  limit  (Q1-1.5(Q3-Q1)  outliers 
beyond  this  range  shown  by  asterisks.  Mean  is  shown  by  circle  with  cross. 


48 


500  -■ 


400  - 


300  - 


200  - 


100  - 


0  - 


AR  Ar2 A,  2  m,  T  arget  Area  1 


MTEKAVG 


jf- 


I 


KRIGEST 


400  -■ 


300  - 


200  - 


100  - 


0  - 


AR  A-2 A,  2  m,  T  arget  Area  2 


MTEKAVG 


I 


KRIGEST 


Figure  36:  Box  and  whisker  plots  for  ARA-2A,  2  m  wide  transect  scenario  comparing 
Mitretek  range  anomaly  densities  (Mtekavg)  and  kriging  estimated  densities  (Krigest)  for 
individual  target  areas.  Boxes  represent  interquartile  range  with  median  shown  by 
horizontal  line.  Vertical  lines  show  largest  value  within  upper  limit  (Q3+1.5(Q3-Q1)  and 
smallest  value  within  lower  limit  (Q1-1.5(Q3-Q1)  outliers  beyond  this  range  shown  by 
asterisks.  Mean  is  shown  by  circle  with  cross. 
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Figure  37:  Box  and  whisker  plots  for  ARA-2A,  2  m  wide  transeet  seenario  eomparing 
Mitretek  range  anomaly  densities  (Mtekavg)  and  kriging  estimated  densities  (Krigest)  for 
individual  target  areas.  Boxes  represent  interquartile  range  with  median  shown  by 
horizontal  line.  Vertieal  lines  show  largest  value  within  upper  limit  (Q3+1.5(Q3-Q1)  and 
smallest  value  within  lower  limit  (Q1-1.5(Q3-Q1)  outliers  beyond  this  range  shown  by 
asterisks.  Mean  is  shown  by  eirele  with  eross. 
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Figure  38:  Box  and  whisker  plots  for  ARA-2A,  2  m  wide  transect  scenario  comparing 
Mitretek  range  anomaly  densities  (Mtekavg)  and  kriging  estimated  densities  (Krigest)  for 

individual  target  areas. 
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Table  11:  Comparison  of  Kriged  estimated  (KrigEst)  and  actual  (Mtk)  magnetic  anomaly 
densities  for  ARA-2A  range.  Estimated  anomaly  densities  were  developed  using  1  m 
wide  transects  (upper  table)  and  2  m  wide  transects  (lower  table). 
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Located  in  the  south-central  area  of  ARA-2A  is  an  impact  area  which  was  not  identified 
in  either  sampling  scenario.  This  target  area  provides  an  example  of  how  a  target  with 
little  ordnance  related  anomalies  in  a  region  with  a  relatively  high  clutter  density  can  go 
undetected.  Although  several  transects  crossed  this  area,  only  a  few  anomalies  were 
detected  (see  Figure  31).  Figure  39  presents  three  plots  showing  the  actual  anomaly 
distribution  for  this  impact  area.  The  top  plot  in  Figure  39  shows  anomalies  due  to  range 
activities  (UXO  and  fragments),  the  middle  plot  shows  the  clutter  anomalies,  and  the 
bottom  plot  shows  the  combination  of  all  anomalies.  A  rectangle  containing  all  range- 
related  anomalies  for  this  impact  area  (black  box  in  Figure  39)  occupies  an  area  of  610 
acres.  Within  this  box  there  are  1,216  range-related  anomalies  (UXO  and  fragments)  and 
5,25 1  clutter  anomalies.  These  give  a  clutter  density  of  8.6  anomalies  per  acre  and  a 
UXO/fragment  density  of  2.0  anomalies  per  acre.  The  ratio  of  range-related  anomaly 
density  to  clutter  density  is  then  0.23. 

For  a  similar  sized  area  (~610  acres)  centered  over  Target  4  which  was  identified  in  the  2 
m  width  transect  sampling,  the  densities  are  4.7  anomalies  per  acre  for  the  range-related 
anomalies  and  6.3  anomalies  per  acre  for  the  clutter.  This  gives  a  range-to-clutter  ratio  of 
0.75  which  is  three  times  that  of  the  unrecognized  impact  area.  The  unrecognized  impact 
area  contains  only  a  small  amount  of  range-related  anomalies,  and  is  located  in  an  area 
with  a  larger  amount  of  clutter.  This  combination  makes  detection  of  this  feature 
difficult.  In  addition,  the  spatial  distribution  of  range-related  fragments  within  this 
impact  area  displays  a  pattern  which  is  unexpected  for  explosive  related  dispersion. 

Close  examination  of  the  top  plot  in  Figure  39  reveals  that  the  range-related  anomalies 
are  concentrated  in  a  cross-pattern  centered  at  the  target  location.  Although  range-related 
anomalies  are  distributed  through-out  the  area,  there  are  notable  linear  concentrations  of 
anomalies  oriented  east-west  and  north-south.  These  concentrations  are  not  explained  by 
the  firing  path  alignment  and  are  observed  at  most  of  the  other  target  locations. 

The  concentration  of  range-related  anomalies  into  a  cross-like  pattern  makes  detection  of 
the  targets  more  difficult  in  that  it  reduces  the  probability  of  encountering  a  high-density 
area  using  linear  transects  (depending  on  orientation)  and  can  result  in  unexpected 
patterns  in  the  detected  anomalies.  Alternate  transect  design  criteria  may  have  been 
employed  to  detect  this  type  of  unexpected  pattern. 
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Figure  39:  Distribution  of  anomalies  for  ARA-2A  impact  area  not  identified  using 
transect  data.  Black-outlined  box  encompasses  all  range  related  anomalies. 
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4.3.2.3  ARA-2A  Power  Curve  Analysis  of  Discrepancies  Between  Designs 

Figure  41  shows  two  VSP  calculated  power  curves  for  the  298  m  (1  m  wide)  and  344  m 
(2  m  wide)  spaced  transect  designs  based  on  the  average  known  maximum  possible  target 
area  size  for  the  simulated  target  areas,  the  known  background  density  for  the  site,  and 
other  inputs  shown  in  Figure  40.  The  maximum  size  of  the  simulated  target  areas  for 
ARA-2A  had  a  slight  oval  shape  with  a  semi-major  radius  ranging  from  865  to  925  m  and 
a  semi-minor  radius  ranging  from  801  to  809  m.  The  two  power  curves  in  Figure  41  are 
based  on  the  average  semi-minor  and  semi-major  radii  for  this  area.  Because  there  is 
very  little  difference  between  the  power  curves  in  Figure  41  and  the  actual  power  curves 
for  each  different  target  area,  one  set  of  power  curves,  which  demonstrates  the  difference 
between  the  two  designs  in  ARA-2A,  is  shown.  On  Figure  41,  these  two  power  curves 
have  reference  points  showing  the  densities  of  the  two  least  dense  target  areas  that  were 
missed.  It  is  no  surprise  that  the  low  density  target  areas  were  not  always  detected  given 
the  probabilities  of  detection  shown  in  Figure  41.  Luckily,  the  344  m  spaced  transect 
design  corresponding  to  the  bottom  power  curve  did  identify  the  target  area  with  a 
density  of  15.47  anomalies  per  acre  (5.35  anomalies  per  acre  above  background).  These 
two  power  curves  are  also  a  function  of  transect  width  and  the  2  m  wide  transect  design 
with  a  wider  spacing  between  transects  had  a  higher  probability  of  detecting  the  actual 
target  sizes  that  were  simulated. 


Figure  40:  Other  inputs  used  to  calculate  power  curves  in  Figure  41 .  Min  Precision  and 
Max  Error  allow  the  user  to  control  the  accuracy  of  the  simulation.  Min  and  Max 
Density  per  acre  sets  the  range  of  the  x  axis  of  the  power  curve 
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Figure  41 :  Two  VSP  calculated  power  curves  for  the  298  m  1  m  wide  (top)  and  344  m  2 

m  wide  (bottom)  spaced  transect  designs 
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4.3.3  Artillery  Range  Area  2B  (ARA-2B) 


Artillery  Range  Area  2B  covered  a  large  area  (37,450  acres).  It  was  also  used  for  155mm 
projectile  and  4.2”  mortar  exercises  and  was  estimated  to  have  a  427  by  280  m  elliptical 
target  area  of  interest.  Two  scenarios  were  created  for  this  impact  area  based  on 
estimating  the  probability  of  detecting  the  defined  target  area  with  a  desired  transect 
density.  Because  the  actual  simulated  target  areas  were  much  larger  than  assumed  for  the 
transect  design  the  probability  of  detecting  the  simulated  target  area  was  much  higher 
than  80%  power  (almost  100%  power)  for  target  areas  with  30  and  50  anomalies  per  acre. 
Thus  an  explanation  of  the  reasoning  for  the  transect  design  based  on  the  assumptions  is 
given  and  then  the  actual  target  area  sizes  are  used  to  show  how  the  transect  design 
performed  as  a  function  of  the  design  power.  Mitretek  simulated  two  different 
background  densities  in  this  site  -  4.05  and  6.07  anomalies  per  acre  (the  actual 
background  clutter  density  was  approximately  2  anomalies  per  acre  instead  of  4.05  for 
the  western  part  of  the  site).  See  Appendix  B  for  a  discussion  about  the  actual 
background  densities  received  from  Mitretek.  The  ARA-2B  background  levels  may  have 
created  some  difficulties  for  identifying  low  density  targets  (targets  with  a  density 
between  0-4  anomalies  per  acre  above  background)  in  areas  where  the  background 
density  was  lower  than  stated.  However,  the  our  performance  on  this  impact  area,  with  a 
fluctuating  background,  did  not  differ  much  from  our  performance  on  the  other  impact 
areas  in  the  simulation. 

4.3.3.1  ARA-2B  Transect  Designs 

Two  transect  designs  based  on  the  ARA-2B  impact  area  were  created.  Each  design  was 
developed  to  have  a  desired  probability  of  detecting  a  certain  target  area  size  and  density. 
As  with  the  two  previous  impact  areas,  the  use  of  4.2”  mortars  and  155mm  projectiles 
created  the  same  assumed  427  m  long  by  280  m  wide  elliptical  target  area. 

For  scenario  1  greater  than  80%  probability  of  identifying  the  assumed  target  area  size 
that  had  a  density  of  50  anomalies  per  acre  or  more  when  the  background  was  6 
anomalies  per  acre  was  desired.  Figure  42  shows  the  power  curve  for  the  344  m  spaced 
transect  design  shown  in  Figure  43  which  covered  0.58%  of  the  impact  area.  The  second 
scenario,  shown  in  Figure  44,  had  a  2  m  wide  282  m  spaced  transect  design  and  covered 
0.71%  of  the  impact  area.  This  design  was  based  on  the  assumptions  used  to  create  the 
power  curve  in  Figure  45.  These  assumptions  were  6  anomalies  per  acre  background 
level  and  greater  than  an  80%  probability  of  detecting  the  assumed  target  area  with  a 
density  of  38  anomalies  per  acre  or  more. 
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Figure  42:  Probability  of  detecting  the  assumed  target  area  with  a  344  m  spaced  transect 
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Figure  43:  The  ARA-2B  344  m  spaced  transect  design  with  anomalies 
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Figure  44:  The  ARA-2B  282  m  spaced  transect  design  with  anomalies 


Figure  45:  Probability  of  detecting  the  assumed  target  area  based  on  the  282  m  spaced 

transect  design. 
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4.3.3.2  ARA-2B  Target  Area  Identification  Results 

The  assumed  target  area  was  mueh  smaller  than  the  actual  target  areas  simulated  in  this 
impact  area.  These  assumed  larger  areas  changed  the  achievable  probabilities  of 
detection  in  the  two  transect  designs.  For  both  designs  the  probability  of  detecting  the 
simulated  target  areas  with  densities  of  50  and  30  anomalies  per  acre  were  over  99.9%. 
Figures  46  and  47  show  the  power  curves  based  on  the  site  background  of  6  anomalies 
per  acre  and  the  average  actual  target  area  sizes  simulated  for  each  transect  design.  The 
average  simulated  target  area  was  a  1856  by  1628  m  ellipse  as  shown  in  Figure  48. 
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Figure  47:  Probability  of  detecting  the  simulated  target  area  based  on  the  344  m  spaced 

transect  design. 


Figure  48:  Target  area  assumptions  tab  with  the  average  target  area  value  for  the  ARA- 

2B  simulated  targets 
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Figures  49  and  5 1  show  the  target  area  perimeter  estimates  for  the  344  m  spaced  transect 
design.  Figure  49  has  the  flagged  areas  of  concern  with  the  joint  PNNL/Sandia  target 
area  perimeter  estimates  (black  boundary  line  around  flagged  areas)  and  the  simulated 
target  areas  color  coded  by  density  above  background.  Figure  51  removes  the  flagging 
and  plots  the  actual  UXO  coordinates  (blue  points).  Similarly,  Figures  50  and  52  show 
the  perimeter  estimates  for  the  282  m  spaced  transect  designs.  In  each  design  the  same 
target  areas  and  approximately  the  same  defined  boundary  shapes  were  identified.  Table 
12  shows  the  target  area  identification  results  for  ARA-2B.  For  both  designs  PNNL  used 
a  900  m  circular  window  and  a  critical  density  of  1 8  anomalies  per  acre. 
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3le  12:  Summary  of  target  area  identi 
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Figure  49:  Target  area  identification  based  on  282  m  spaced  transects  design  with  actual 
target  areas  identified  and  color  coded  by  density  above  background 
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Figure  50:  Target  area  identification  based  on  344  m  spaced  transects  design  with  actual 
target  areas  identified  and  color  coded  by  density  above  background 
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Figure  51:  UXO  locations  (blue  clusters)  for  the  282  m  spaced  transect  design  boundaries 
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Figure  52:  UXO  locations  (blue  clusters)  for  the  344  m  spaced  transect  design 

boundaries 

For  the  Indicator  Kriging  analysis  of  ARA-2B,  SNL  used  a  critical  density  of  30 
anomalies  per  acre  for  both  transect  designs.  As  before,  Sandia  identified  the  targets 
using  a  rectangular  averaging  window  350  m  in  length  with  a  width  equal  to  the  transect 
sample  width,  in  this  case  2  meters. 

Figure  53  shows  the  Indicator  Kriging  probability  levels  for  ARA-2B  using  an  Indicator 
Kriging  threshold  of  30  anomalies  per  acre  for  the  282  m  and  345  m  spaced-transects. 
Only  areas  with  a  probability  of  0.05  or  greater  of  exceeding  the  threshold  are  shown  by 
color-filled  contours.  These  areas  were  included  as  part  of  the  information  used  in 
determining  the  final  target  boundaries.  Variogram  model  parameters  used  in  the  kriging 
estimates  are  listed  in  Appendix  A. 

The  Indicator  Kriging  results  for  both  transect  designs  show  several  small  isolated  areas 
outside  of  the  final  target  delineations.  These  were  considered  too  small  to  be  actual 
target  sites  and  so  were  not  incorporated  into  the  final  target  boundaries.  In  general,  the 
target  areas  delineated  by  the  Indicator  Kriging  are  similar  between  the  two  sampling 
scenarios.  For  Target  Area  2  there  is  a  notable  difference  along  the  north-eastern 
boundary.  For  the  345  m  spaced  transects,  this  margin  extends  further  to  the  north-east 
outside  of  the  simulated  impact  area.  The  boundary  for  the  282  m  spaced  transects  more 
closely  follows  the  impact  area  boundary  indicating  that  the  finer  transect  spacing  is 
acting  to  constrain  the  boundary  because  of  the  additional  sampling  locations. 
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Figure  53:  Indicator  kriging  probability  levels  for  ARA-2B.  Top  figure  shows  levels  for 
282  m  spaced  transects;  bottom  figure  shows  levels  for  345  m  spaced  transects.  Numbers 

indicate  final  target  area  identifiers. 
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Figures  54  and  55  show  the  kriged  estimated  anomaly  densities  and  the  cell-averaged  true 
anomaly  densities  for  the  282  and  345  m  transect  sampling  scenarios  respectively.  Both 
figures  show  that  the  kriged  estimates  represent  the  actual  anomaly  density  distribution 
very  well.  In  fact,  many  of  the  individual  high-density  target  locations  depicted  in  the 
actual  anomaly  plots  are  easily  discernible  in  the  kriged  estimates  from  both  sampling 
scenarios.  The  very  low  density  targets  shown  in  Figure  50  do  not  appear  in  Figure  54 
which  shows  cell-averaged  density  levels  for  the  kriged  density  levels  and  the  actual 
density  levels. 

The  similarity  between  the  actual  and  estimated  anomaly  densities  are  also  reflected  in 
comparative  box  and  whisker  plots  of  the  data  (Figures  56  through  58),  and  in  the 
descriptive  statistics  from  the  data  (Table  13).  As  shown  by  the  comparative  box  and 
whisker  plots,  the  interquartile  ranges  and  means  of  the  kriged  estimates  are  very  similar 
to  the  actual  cell-averaged  anomaly  data.  The  means  from  both  scenarios  differ  from  the 
actual  mean  by  less  than  9%. 

Not  surprisingly,  the  results  from  the  282  m  transect  spacing  scenario  match  the 
characteristics  of  the  original  data  set  more  closely  than  the  coarser  sampling  scenario. 
The  increased  sampling  of  this  scenario  reduces  the  difference  in  mean  values  between 
the  actual  and  estimated  data  sets  to  less  than  3%. 
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Figure  54:  Magnetic  anomaly  densities  in  anomalies  per  acre  for  ARA-2B.  Top  figure 
shows  kriged  density  levels  developed  using  282  m  spaced  transects;  bottom  figure 
shows  actual  cell-averaged  density  levels. 
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Figure  55:  Magnetic  anomaly  densities  in  anomalies  per  acre  for  ARA-2B.  Top  figure 
shows  kriged  density  levels  developed  using  345  m  spaced  transects;  bottom  figure 
shows  actual  cell-averaged  density  levels. 
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Figure  56:  Box  and  whisker  plots  for  ARA-2B  comparing  Mitretek  range  anomaly 
densities  (Mtekavg)  and  kriging  estimated  densities  (Krigest)  for  individual  target  areas. 
Kriging  estimates  developed  using  282  m  spaced  transects.  Boxes  represent  interquartile 
range  with  median  shown  by  horizontal  line.  Vertical  lines  show  largest  value  within 
upper  limit  (Q3+1.5(Q3-Q1)  and  smallest  value  within  lower  limit  (Q1-1.5(Q3-Q1) 
outliers  beyond  this  range  shown  by  asterisks.  Mean  is  shown  by  circle  with  cross. 
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Figure  57:  Box  and  whisker  plots  for  ARA-2B  comparing  Mitretek  range  anomaly 
densities  (Mtekavg)  and  kriging  estimated  densities  (Krigest)  for  individual  target  areas. 
Kriging  estimates  developed  using  345  m  spaced  transects.  Boxes  represent  interquartile 
range  with  median  shown  by  horizontal  line.  Vertical  lines  show  largest  value  within 
upper  limit  (Q3+1.5(Q3-Q1)  and  smallest  value  within  lower  limit  (Q1-1.5(Q3-Q1) 
outliers  beyond  this  range  shown  by  asterisks.  Mean  is  shown  by  circle  with  cross. 
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Figure  58:  Box  and  whisker  plots  for  ARA-2B  eomparing  Mitretek  range  anomaly 
densities  (Mtekavg)  and  kriging  estimated  densities  (Krigest)  for  entire  range  site.  Top 
plot  shows  kriging  estimates  developed  using  282  m  spaeed  transeets;  bottom  plot  shows 
estimates  using  345  m  spaeed  transeets.  Boxes  represent  interquartile  range  with  median 
shown  by  horizontal  line.  Vertieal  lines  show  largest  value  within  upper  limit 
(Q3+1.5(Q3-Q1)  and  smallest  value  within  lower  limit  (Q1-1.5(Q3-Q1)  outliers  beyond 
this  range  shown  by  asterisks.  Mean  is  shown  by  eirele  with  eross. 
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Area 

Source 

Mean 

Mean  Diff. 

Std  Dev 

Min 

Max 

282  m  spaced 
Transects 

Target  1 

Mtk 

5E4 

3.3% 

53.5 

0 

550.4 

KrigEst 

53.1 

45.8 

0 

341.0 

Target  2 

Mtk 

29.5 

-5.1% 

24.9 

0 

236.3 

KrigEst 

28.0 

19.8 

0 

109.8 

Entire  Range 

Mtk 

8.2 

-2.4% 

16.7 

0 

550.4 

KrigEst 

8.0 

15.5 

0 

341.0 

Area 

Source 

Mean 

Mean  Diff. 

Std  Dev 

Min 

Max 

345  m  spaced 
Transects 

Target  1 

Mtk 

53.6 

-8.2% 

54.0 

0 

550.4 

KrigEst 

49.2 

38.5 

0 

242.8 

Target  2 

Mtk 

26.5 

-8.3% 

23.9 

0 

236.3 

KrigEst 

24.3 

20.3 

0 

185.0 

Entire  Range 

Mtk 

8.2 

-8.5% 

16.7 

0 

550.4 

KrigEst 

7.5 

13.8 

0 

242.8 

Table  13:  Comparison  of  Kriged  estimated  (KrigEst)  and  actual  (Mtk)  magnetic  anomaly 
densities  for  ARA-2B  range.  Estimated  anomaly  densities  were  developed  using  282  m 
spaced  transects  (upper  table)  and  345  m  spaced  transects  (lower  table). 


4.3.4  Suspected  Range  Area  1  (SRA-1) 

Suspected  Range  Area  1  covered  6,200  acres  and  was  used  for  small  arms  training. 
While  it  is  suspected  that  there  were  20mm  and  105mm  munitions  used  in  this  impact 
area,  the  transect  designs  were  based  the  target  area  assumptions  of  a  105mm  site. 

4.3.4.1  SRA-1  Transect  Design 

The  first  transect  design  had  a  438  m  square  grid  spacing  between  each  2  m  transect  to 
have  a  95%  probability  of  traversing  a  398  by  250  m  elliptical  target  area  that  covered 
0.91%  of  the  impact  area.  The  second  design  was  developed  to  traverse  a  smaller  target 
than  the  assumed  105mm  target  area.  This  design  had  a  95%  probability  of  traversing  a 
198  X  62  m  potential  target  area  which  resulted  in  a  204  m  square  grid  spaced  transect 
design  that  covered  1.94%  of  the  impact  area.  Figure  59  shows  the  two  transect  designs 
implemented  for  this  area. 
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4.3. 4.2  SRA-1  Target  Identification  Results 


After  performing  target  detection  and  delineation  tests,  for  both  designs,  no  additional 
transects  were  requested  and  no  potential  target  areas  were  identified.  This  was  reported 
to  Mitretek  and  they  confirmed  that  the  simulation  for  this  area  was  designed  to  have 
background  clutter  only  but  no  used  targets.  Therefore,  the  PNNL/SNL  conclusion  that 
there  were  no  target  areas  on  this  site  was  correct. 

The  information  on  the  SRA-1  area  anomaly  distribution  as  shown  in  right  side  of  Figure 
60  was  used  to  conclude  that  there  were  no  identifiable  target  areas.  Figure  60  shows  the 
histogram  of  the  window  densities  that  are  calculated  from  a  moving  window  centered  on 
each  traversed  transect  that  calculates  the  density  of  anomalies  per  area  of  transects  lying 
within  the  user  defined  radius  of  the  circular  window.  In  comparison,  the  left  side  of 
Figure  60  shows  the  histogram  for  the  window  densities  based  on  ARA-2A  where  target 
areas  did  exist.  This  histogram  shows  the  combination  of  the  distributional  patterns  of 
background  with  additional  distributions  based  on  different  target  areas  that  were  in 
ARA-2A  (See  Section  4.3.2).  The  histogram  from  SRA-1  has  a  much  different  shape 
from  any  of  the  other  impact  areas  in  this  simulation  because  it  only  identified  one 
distribution  instead  of  a  mixture  of  multiple  distributions.  The  histogram  to  the  right  in 
Figure  60  and  the  kriged  anomaly  density  maps  in  Figure  61,  which  are  informative  in 
that  they  show  the  type  of  spatial  distribution  resulting  from  a  non-use  area,  signaled  that 
this  site  did  not  contain  identifiable  target  areas. 


'  Vliud  ^  (Mlnlirt  4  SU  1  ^  tDniin 

■9_*  -  -  rra^an  y _ 


Figure  59:  Sparse  transect  design  for  SRA-1  based  on  a  396  by  250  m  target  area  (left) 
and  compact  transect  design  for  SRA-1  based  on  a  198  by  62  m  target  area  (right). 
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Figure  60:  Anomaly  density  histograms.  The  histogram  to  the  left  is  based  on  the  dense 
transect  design  scenario  in  ARA-2A.  The  histogram  to  the  right  is  developed  using  the 
20mm  SRA-1  scenario. 


Figure  61:  Kriged  anomaly  density  maps  in  anomalies  per  acre  for  SRA-1  site.  Left  plot 
created  using  438  m  spaced  transects;  right  plot  created  using  204  m  spaced  transects. 
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4.3.5  Bombing  and  Gunnery  Range  (BGR-1) 

The  Bombing  and  Gunnery  Range  covered  74,650  acres  and  was  the  largest  impact  area 
with  live  and  dummy  munitions  used  in  air-to-ground  training.  Previous  information  did 
indicate  that  there  were  potential  bomb  dumps  located  in  this  impact  area  along  with 
random  target  areas  used  for  100,  300,  500,  1000,  and  2000  lb.  bomb  exercises.  No 
specific  location  information  for  either  the  bomb  dumps  or  the  targets  was  provided. 
Three  different  scenarios  (transect  designs)  for  this  impact  area  were  developed. 
Preliminary  results  from  each  design  signaled  that  the  assumed  target  areas  were  much 
larger  than  what  had  been  simulated  on  the  site.  Thus  scenario  1  was  terminated  and 
additional  transects  over  the  entire  site  for  scenarios  2  and  3  were  requested. 

4.3.5.1  BGR-1  Transect  Design 

Scenario  1  was  based  on  an  estimated  2438  by  1625  m  elliptical  fragment  dispersion  of  a 
2000  lb.  bomb  and  a  95%  probability  of  traversing  random  orientation  target  areas. 

Based  on  the  first  analysis  of  the  anomaly  data  observed,  it  became  apparent  that  the  2  m 
parallel  transects  spaced  1989  m  apart  were  too  sparse  for  the  identified  potential  target 
areas.  Those  target  areas  that  were  identified  were  definitely  much  smaller  than  what 
was  originally  assumed  under  this  design  scenario.  Therefore,  no  additional  work  was 
done  for  this  scenario. 

Scenario  2  assumed  a  914  by  320  m  elliptical  fragment  dispersion  based  on  a  500  lb. 
bomb  and  a  95%  probability  of  traversing  random  orientation  target  areas  which  resulted 
in  2  m  parallel  transects  spaced  461  m  apart.  After  an  initial  analysis  of  the  resulting 
anomaly  data,  several  small  target  areas  were  identified  so  another  set  of  parallel 
transects  spaced  461  m  apart  were  requested  to  fall  between  each  of  the  original  transects. 
Thus,  the  final  transect  spacing  was  approximately  230  m  apart.  The  complete  set  of 
transects  covered  0.86%  of  the  area  and  is  shown  in  Figure  62  with  the  identified 
anomalies. 

Scenario  3,  shown  in  Figure  63,  was  similar  to  scenario  2  with  the  exception  of  assuming 
that  the  orientation  of  the  targets  was  known.  The  bomb  delivery  approach  was  assumed 
to  be  parallel  to  the  long  axis  of  the  impact  area.  This  change  in  assumptions  resulted  in 
2  m  wide  parallel  transects  spaced  962  m  apart.  Again,  after  an  initial  analysis  of  the 
resulting  anomaly  data,  several  small  target  areas  were  identified  so  additional  2  m 
transects  spaced  482  m  apart  which  traveled  perpendicular  to  the  original  transects  were 
requested.  The  complete  transect  request  covered  1.2%  of  the  impact  area  (Figure  63). 
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Figure  62:  Scenario  2  transect  design.  The  red  lines  represent  the 
additional  transects  requested. 


75 


^TlSijalBanip!^^!ar^TB[!IS9?58^SHvB  In  one  MltnelSUTv^^ 

^  Bk  Utp  t'Al  £air««iii||  Sfifilt  5f>tl>Sfta  figom  ^wwi  JAChhIvw  U«l|i 

Qji^iaiisiai  \  :^iBimiai]|~i|M  f  i  v 


^72000  674000  CTSOM  OISflOO  CgO"CiO  6e?00O 

f<»r  H(^^K  pf«S  Fi 


690000 


osnooo  oagooo 


Figure  63:  Scenario  3  transect  design.  The  red  (north  south)  lines  represent  the 

additional  transects  requested. 


4.3.5.2  BGR-1  Target  Area  Identification  Results 

The  complete  scenario  2  design  was  used  to  flag  the  final  potential  target  areas.  Figure 
64  shows  actual  target  area  sizes  colored  by  their  respective  densities  with  the  flagged 
areas  and  target  area  perimeter  shapes  identified.  As  stated  before,  the  target  area  shapes 
were  a  combination  of  PNNL’s  flagging  routine  and  Sandia’s  target  delineation  tools. 
PNNL  used  a  600  m  diameter  window  with  a  critical  density  of  25  anomalies  per  acre  to 
flag  their  potential  target  areas.  Similar  to  the  other  ranges,  Sandia  identified  their  targets 
using  a  rectangular  averaging  window  350  m  in  length  with  a  width  equal  to  the  transect 
sample  width  of  2  meters.  For  its  Indicator  Kriging  analysis,  SNL  used  a  critical  density 
of  55  anomalies  per  acre  for  scenario  2  of  BGR-1. 
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The  final  analysis  for  scenario  3  was  based  on  the  rectangular  grid  transects  design. 

Figure  65  shows  the  transect  design,  the  actual  simulated  target  areas  color  coded  by 
density,  and  the  boundaries  determined  by  a  combination  of  PNNL’s  flagging  routine  and 
Sandia’s  target  delineation  tools.  This  design  had  a  transect  that  passed  directly  through 
the  middle  of  the  100  m  radius  target  area.  PNNL  used  a  600  m  diameter  window  and  a 
40  anomalies  per  acre  critical  density  to  identify  the  potential  target  areas.  SNL’s 
procedure  was  similar  to  that  used  for  scenario  2.  A  critical  density  threshold  of  55 
anomalies  per  acre  was  used  for  the  Indicator  Kriging. 

The  actual  target  area  sizes  and  densities  simulated  for  this  impact  area  were  much 
different  than  expected.  The  densities  of  each  target  area  in  BGR-1  range  from  48 
anomalies  per  acre  to  687  anomalies  per  acre  with  a  background  density  of  8.09 
anomalies  per  acre.  Thus,  this  site  did  not  have  the  difficult  task  of  identifying  low 
density  target  areas.  The  unique  feature  of  this  site  was  the  varying  size  of  each  of  the 
target  areas.  They  ranged  from  a  600  m  radius  circle  to  a  100  m  radius  circle  and  there 
were  five  targets  with  a  radius  smaller  than  200  meters.  Based  on  the  used  transect 
design  there  was  a  high  likelihood  that  it  would  not  traverse  the  100  m  radius  circle  with 
either  scenario  2  or  3  transect  designs.  All  target  areas  that  had  a  diameter  larger  than  the 
transect  design  spacing  were  identified.  Figure  66  shows  an  example  of  one  of  the 
simulated  targets  in  BGR-1  with  the  blue  dots  representing  UXO  placement  and  Table  14 
summarizes  the  target  identification  results. 


Density  of 
Target  Areas 
above 
background 

Transect 

Spacing 

Percent  of 
Site 

Covered 

with 

Transects 

Totai 

Number 

of 

Target 

Areas 

Total 
Number 
of  UXO 

Number 
of  Target 
Areas 
Identified 

Number  of 
UXO 

Contained 

Percent 
of  Target 
Areas 
Identified 

Percent  of 
UXO 

Contained 

>6 

232m 

0.86% 

24 

92049 

24 

91404 

100.00% 

99.30% 

>6 

483m  X  961m 

1 .20% 

24 

92049 

24 

92049 

100% 

100% 

<6 

483m  X  961m 

1 .20% 

No  target  areas  with  density  less  than  6  per  acre. 

<6 

232m 

0.86% 

No  target  areas  with  density  less  than  6  per  acre. 

Table  14:  Summary  of  the  BGR-1 


arget  Area  Identification  Results 


Figure  67  shows  the  indicator  kriging  probability  levels  for  BGR-1  scenarios  2  and  3 
using  indicator  kriging  thresholds  of  55  anomalies  per  acre  for  both  scenarios.  Only 
areas  with  a  probability  of  0.05  or  greater  are  shown  by  color-filled  contours.  These 
areas  were  included  as  part  of  the  information  used  in  determining  the  final  target 
boundaries.  Variogram  model  parameters  used  in  the  kriging  estimates  are  listed  in 
Appendix  A. 


As  seen  in  Figure  67,  the  results  from  both  sampling  designs  are  similar.  The  major 
difference  between  the  target  delineations  from  the  two  scenarios  is  a  single  target  area 
identified  in  scenario  3  that  was  not  detected  in  scenario  2.  Figures  68  and  69  present 
kriging  estimates  of  anomaly  densities  for  scenarios  2  and  3  respectively.  The  cell 
averaged  actual  anomaly  densities  for  BGR-1  are  also  presented  for  comparison.  As 
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Figures  68  and  69  show,  the  kriged  estimates  of  anomaly  densities  developed  from  the 
limited  transect  data  closely  match  the  spatial  distribution  of  the  actual  anomaly  data  set, 


Figure  64:  BGR-1  Scenario  2  transect  design  with  flagged  areas  of  concern,  joint 
PNNL/Sandia  target  area  perimeters,  and  simulated  target  areas  colored  by  density  above 

background 
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Figure  65:  BGR-1  Scenario  3  transect  design  with  flagged  areas  of  concern,  joint 
PNNL/Sandia  target  area  perimeters,  and  simulated  target  areas  colored  by  density  above 

background 
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Figure  66:  Example  of  UXO  placement  in  BGR-1  simulated  target  areas.  The  green 
circle  represents  the  simulated  target  area  boundary  and  the  black  line  represents  the 

perimeter  delineation. 


Tables  15  and  16  present  summary  descriptive  statistics  for  the  BGR-1  range  and  each  of 
the  identified  target  areas  for  scenarios  2  and  3  respectively.  As  shown  by  these  tables, 
the  difference  in  the  means  between  actual  and  estimated  anomaly  densities  is  less  than 
3%  for  the  entire  site  for  each  scenario.  However,  the  difference  in  means  for  individual 
target  areas  was  as  high  as  38%  for  scenario  2  and  56%  for  scenario  3. 

Further  investigation  into  the  areas  with  large  differences  between  the  estimated  and 
actual  anomaly  densities  revels  that  these  discrepancies  are  primarily  due  to  the  small 
size  of  the  actual  target  areas  with  respect  to  the  sample  transect  spacing.  For  example, 
target  area  6  had  the  largest  difference  in  means  for  scenario  3  (55.6%)  but  a  much 
smaller  difference  for  scenario  2  (1 1.5%).  Three  of  the  sampling  transects  for  scenario  2 
passed  through  the  actual  fragmentation  area  for  that  target,  whereas  only  one  of  the 
transects  from  scenario  3  sampled  the  actual  target  area.  A  similar  pattern  is  seen  in  the 
other  target  areas  with  large  differences  in  the  means. 
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Figure  67:  Indicator  kriging  probability  levels  for  BGR-1.  Top  figure  shows  levels  for 
scenario  2;  bottom  figure  shows  levels  for  scenario  3.  Numbers  indicate  final  target 
identifiers.  Maps  rotated  from  original  coordinate  system. 
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Figure  68:  Magnetic  anomaly  densities  in  anomalies  per  acre  for  BGR-1.  Top  figure 
shows  kriged  density  levels  for  scenario  2;  bottom  figure  shows  actual  cell-averaged 
density  levels.  Maps  rotated  from  original  coordinate  system. 
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Figure  69:  Magnetic  anomaly  densities  in  anomalies  per  acre  for  BGR-1.  Top  figure 
shows  kriged  density  levels  for  scenario  3;  bottom  figure  shows  actual  cell-averaged 
density  levels.  Maps  rotated  from  original  coordinate  system. 
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Area 

Source 

Mean 

Mean  Diff. 

Std  Dev 

Min 

Max 

Target  1 

Mtk 

51.0 

7.8% 

134.9 

0 

1617.1 

KrigEst 

55.0 

55.9 

0 

222.4 

Target  2 

Mtk 

88.4 

-12.0% 

171.0 

0 

2734.1 

KrigEst 

77.8 

79.4 

0 

301.6 

Target  3 

Mtk 

65.4 

37.5% 

123.3 

0 

830.4 

KrigEst 

89.9 

104.7 

0 

399.1 

Target  4 

Mtk 

70.3 

-1 .7% 

83.2 

0 

464.6 

KrigEst 

69.1 

64.4 

0 

285.0 

Target  5 

Mtk 

63.4 

-3.0% 

92.1 

0 

464.6 

KrigEst 

61.5 

56.7 

0 

225.7 

Target  6 

Mtk 

92.4 

1 1 .5% 

167.6 

0 

916.2 

KrigEst 

103.0 

134.9 

0 

583.6 

Target  7 

Mtk 

79.4 

-1 .6% 

137.5 

0 

861.2 

KrigEst 

78.1 

94.7 

0 

461.1 

Target  8 

Mtk 

90.7 

-7.4% 

145.0 

0 

893.6 

KrigEst 

84.0 

98.6 

0 

420.2 

Target  9 

Mtk 

90.9 

7.8% 

127.1 

1.6 

600.6 

KrigEst 

98.0 

108.1 

0 

412.6 

Target  10 

Mtk 

81.1 

10.0% 

109.0 

0 

642.6 

KrigEst 

89.2 

100.4 

0 

487.0 

Target  1 1 

Mtk 

76.3 

21.4% 

159.4 

0 

1138.0 

KrigEst 

92.6 

128.2 

0 

573.5 

Target  12 

Mtk 

71.3 

3.8% 

117.2 

0 

874.1 

KrigEst 

74.0 

82.9 

0 

391.3 

Target  13 

Mtk 

85.6 

2.7% 

165.4 

0 

1178.5 

KrigEst 

87.9 

110.9 

0 

501.5 

Target  14 

Mtk 

45.2 

-8.6% 

44.4 

0 

244.4 

KrigEst 

41.3 

31.0 

5.4 

148.0 

Target  15 

Mtk 

97.2 

-36.8% 

221.7 

1.6 

1701.3 

KrigEst 

61.4 

71.7 

0 

283.0 

Target  16 

Mtk 

69.1 

1 1 .3% 

110.1 

0 

937.3 

KrigEst 

76.9 

83.8 

0 

367.3 

Target  17 

Mtk 

57.9 

10.5% 

48.6 

4.9 

249.3 

KrigEst 

64.0 

48.2 

6.9 

197.7 

Target  18 

Mtk 

56.0 

19.6% 

87.3 

0 

461.3 

KrigEst 

67.0 

69.1 

0.4 

291.3 

Target  19 

Mtk 

78.9 

17.4% 

162.3 

0 

118.6 

KrigEst 

92.6 

116.8 

0 

487.7 

Target  20 

Mtk 

114.4 

-32.1 

186.3 

0 

953.4 

KrigEst 

77.7 

80.5 

0 

281.3 

Target  21 

Mtk 

107.4 

-18.6 

313.3 

0 

2993.1 

KrigEst 

87.7 

107.3 

0 

422.0 

Entire  Range 

Mtk 

12.7 

1.6% 

39.97 

0 

2993.06 

KrigEst 

12.9 

29.62 

0 

583.60 

Table  15:  Comparison  of  Kriged  estimated  (KrigEst)  and  actual  (Mtk)  magnetic 


anomaly  densities  for  BGR-1  range  scenario  2. 
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Area 

Source 

Mean 

Mean  Diff. 

Std  Dev 

Min 

Max 

Target  1 

Mtk 

53.0 

6.6% 

137.9 

0 

1617.1 

KrigEst 

56.5 

56.9 

0 

222.4 

Target  2 

Mtk 

66.8 

-1 1 .4% 

150.7 

0 

2734.1 

KrigEst 

59.2 

74.7 

0 

301.6 

Target  3 

Mtk 

40.5 

35.1% 

96.9 

0 

830.4 

KrigEst 

54.7 

88.3 

0 

399.1 

Target  4 

Mtk 

51.7 

-20.3% 

75.4 

0 

464.6 

KrigEst 

41.2 

31.1 

0 

146.6 

Target  5 

Mtk 

61.1 

38.3% 

90.8 

0 

464.6 

KrigEst 

84.5 

84.2 

0 

343.8 

Target  6 

Mtk 

95.7 

55.6% 

170.2 

0 

916.2 

KrigEst 

148.9 

143.9 

0 

580.4 

Target  7 

Mtk 

69.7 

16.5% 

130.1 

0 

861.2 

KrigEst 

81.2 

103.3 

0 

574.5 

Target  8 

Mtk 

72.3 

-4.1% 

132.5 

0 

893.6 

KrigEst 

69.3 

58.7 

0 

227.6 

Target  9 

Mtk 

68.4 

37.9% 

114.8 

0 

600.6 

KrigEst 

94.3 

124.8 

0 

633.0 

Target  10 

Mtk 

79.9 

2.1% 

105.7 

0 

642.6 

KrigEst 

81.6 

90.4 

0 

473.8 

Target  1 1 

Mtk 

65.3 

19.6% 

148.2 

0 

1138.0 

KrigEst 

78.1 

122.0 

0 

573.5 

Target  12 

Mtk 

70.1 

6.6% 

116.5 

0 

874.1 

KrigEst 

74.7 

84.2 

0 

436.3 

Target  13 

Mtk 

85.5 

1 .6% 

165.3 

0 

1178.5 

KrigEst 

86.9 

111.3 

0 

501.5 

Target  14 

Mtk 

42.1 

8.1% 

45.3 

0 

244.4 

KrigEst 

45.5 

33.5 

5.8 

159.8 

Target  15 

Mtk 

73.6 

-37.1% 

194.0 

1.6 

1707.3 

KrigEst 

46.3 

66.5 

0 

283.0 

Target  16 

Mtk 

79.9 

-8.4% 

117.3 

0 

937.3 

KrigEst 

73.2 

63.6 

0.1 

265.0 

Target  17 

Mtk 

39.0 

16.2% 

43.3 

0 

249.3 

KrigEst 

45.3 

36.0 

0.4 

143.9 

Target  18 

Mtk 

53.2 

-44.4% 

85.2 

0 

461.3 

KrigEst 

29.6 

18.3 

0.7 

73.7 

Target  19 

Mtk 

159.9 

9.4% 

227.7 

1.6 

1118.6 

KrigEst 

175.0 

140.4 

6.0 

487.7 

Target  20 

Mtk 

33.6 

-49.1% 

108.9 

0 

1044.1 

KrigEst 

17.1 

12.0 

0 

42.0 

Target  21 

Mtk 

92.5 

25.6% 

171.4 

0 

953.4 

KrigEst 

116.2 

125.1 

0.4 

506.6 

Target  22 

Mtk 

155.6 

-27.6% 

372.7 

0 

2993.1 

KrigEst 

112.6 

121.9 

0 

422.0 

Entire  Range 

Mtk 

12.7 

2.4% 

42.1 

0 

2993.1 

KrigEst 

13.0 

31.1 

0 

583.6 

Table  16:  Comparison  of  Kriged  estimated  (KrigEst)  and  actual  (Mtk)  magnetic 


anomaly  densities  for  BGR-1  range  scenario  3. 
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5.  Summary  of  Data  Analysis,  Interpretation  and  Evaluation  for  the 

SimRangE  4  Simulation 

Ordnance  operations  were  simulated  on  five  different  ranges  by  Mitretek  and  the  results 
of  these  simulations  were  kept  from  the  PNNL/Sandia  statistical  site  characterization 
design  tools  team.  For  each  of  the  simulated  ranges,  the  suspected  ordnance  types  were 
provided  to  the  characterization  design  team,  and  for  one  of  the  ranges  a  suspected 
location  and  target  diameter  were  also  provided.  Based  on  this  information,  the 
PNNL/SNL  team  assumed  various  conceptual  site  models  (CSM)  for  each  range  and 
requested  geophysical  sampling  transects  where  the  transect  designs  were  based  on  each 
CSM.  The  locations  of  the  geophysical  anomalies  along  the  requested  transects  was 
provided  back  to  the  PNNL/SNL  team  by  Mitretek.  Based  on  these  results,  target 
boundaries  and  anomaly  density  maps  were  created.  In  some  cases,  additional  transects 
were  requested  to  better  define  the  existence  and/or  the  extent  of  suspected  target  areas. 

5.1  Data  Analysis  Results 

The  performance  of  the  PNNL/SNL  methods  was  excellent  with  100%  of  the  UXO  and 
100%  of  the  target  areas  that  had  a  density  as  low  as  6  anomalies  per  acre  above 
background  identified  (with  the  exception  of  the  sparse  BGR-1  design).  One  target  area 
in  the  BGR-1  impact  area  had  a  diameter  of  200  m.  Even  though  the  transect  spacing  for 
both  BGR-1  designs  were  spaced  farther  apart  than  the  200  m  diameter  target  area,  one  of 
the  two  transect  designs  did  identify  this  target  area. 

With  exception  to  ARA-1,  all  of  the  impact  areas  had  a  sparse  and  a  dense  transect 
design.  The  multiple  transect  designs  were  created  to  show  the  sensitivity  of  target 
identification  to  transect  spacing.  Due  to  the  large  differences  in  assumed  target  area 
sizes  to  the  simulated  target  area  sizes,  ARA-2B  and  BGR-1  transect  designs  did  not 
provide  the  type  of  scenarios  where  this  sensitivity  could  be  examined.  SRA-1  had  two 
transect  designs  but  no  targets  were  located  in  this  impact  area  and  only  one  transect 
design  was  requested  for  ARA-1.  However,  ARA-2A  did  provide  differing  results  on  the 
simulated  targets  based  on  the  transect  design.  It  is  not  clear  that  the  differing  results  for 
the  Ft.  Ritchie  target  data  were  a  result  of  the  transect  design  or  the  unique  way  in  which 
these  data  were  applied  to  the  site.  ARA-2A  showed  that  the  tighter  1  m  transects  did  not 
perform  as  well  as  the  slightly  sparser  2  m  transects.  Figure  32  showed  the  power  curves 
that  help  explain  these  results.  For  the  ARA-2A  site,  the  Ft.  Ritchie  data  was  reported 
separately  and  will  not  be  included  in  the  combined  summary  tables  (Tables  13  and  14). 

Overall,  the  kriged  estimates  of  magnetic  anomaly  densities  compared  well  with  the  true 
density  values.  For  those  target  areas  delineated  in  ranges  ARA-1,  ARA-2A  and  ARA- 
2B,  for  all  scenarios,  the  average  value  of  the  absolute  values  of  mean  difference  between 
the  true  and  estimated  anomaly  densities  was  only  7.6%.  This  value  drops  to  3.5%  when 
computed  using  range-wide  mean  difference  values.  These  values  indicate  that  the 
estimated  anomaly  densities  matched  the  true  density  values  closely  for  these  ranges. 


86 


BGR-1  contains  the  delineated  target  areas  with  the  largest  differences  in  means  of 
anomaly  densities  for  the  delineated  target  areas.  Scenario  2  for  BGR-1  eontained  21 
identified  target  areas,  4  of  these  had  mean  differenees  between  the  true  and  estimated 
magnetie  anomaly  densities  that  exeeeded  20%.  Seenario  3  for  BGR-1  eontained  22 
identified  target  areas,  10  of  whieh  had  mean  differences  exeeeding  20%.  The  average 
value  of  the  absolute  values  of  the  mean  differences  for  the  delineated  target  areas  is 
13.5%  for  seenario  2  and  21%  for  scenario  3.  The  spatial  eharacteristics  of  the  small, 
high-density  targets  of  BGR-1  were  not  well  represented  by  the  coarser  spaeing  of  the 
Seenario  3  transects.  This  lead  to  the  situation  where,  although  the  targets  were  deteeted, 
the  eompaet  spatial  strueture  of  the  targets  eould  not  be  recreated  through  the  limited 
transect  data.  Additional  transect  data  would  be  necessary  to  aceurately  reflect  the  spatial 
structure  of  the  relatively  small  targets  contained  in  BGR-1. 

Overall,  the  results  from  the  magnetic  anomaly  density  mapping  exereise  show,  that  with 
only  limited  transeet  information,  reliable  estimates  of  the  magnetie  anomaly  densities 
can  be  developed.  A  majority  of  the  identified  target  areas  had  mean  values  of  anomaly 
density  that  differed  from  the  true  value  by  less  than  20%.  Although  the  means  of  some 
of  the  identified  targets  in  BGR-1  had  differences  greater  than  20%,  the  general  pattern  of 
the  true  anomaly  distributions  were  well  represented. 

The  final  summary  is  based  on  the  most  eonservative  designs  for  each  impact  area.  Table 
17  summarizes  the  results  for  each  scenario  based  on  target  areas  that  had  a  density 
greater  than  6  anomalies  per  acre  above  baekground  and  Table  18  eontains  the 
demonstration  performanee  objeetives. 


Impact 

Area 

T  ransect 
Spacing 

Density  of 
Target 
Areas 
above 
background 

Percent 
of  Site 
Covered 
with 

Transects 

Total 

Number 

of 

Target 

Areas 

Total 
Number 
of  UXO 

Number 
of  Target 
Areas 
Identified 

Number 
of  UXO 
Contained 

Percent 
of  Target 
Areas 
Identified 

Percent  of 
UXO 

Contained 

ARA-2B 

282m 

>6 

0.71% 

9 

253 

9 

253 

100.00% 

100.00% 

ARA-2A 

344m 

>6 

0.60% 

3 

175 

3 

175 

100.00% 

100.00% 

SRA-1 

204m  X  204m 

>6 

1 .94% 

None 

None 

None 

None 

Correct 

Correct 

ARA-1 

344m 

>6 

0.66% 

7 

110 

7 

110 

100.00% 

100.00% 

BGR-1 

483m  X  961m 

>6 

1 .20% 

24 

92049 

24 

92049 

100.00% 

100.00% 

ARA-2B 

282m 

<6 

0.71% 

8 

19 

1 

0 

12.50% 

0.00% 

ARA-2A 

344m 

<6 

0.60% 

2 

38 

1 

27 

50.00% 

71 .05% 

SRA-1 

204m  X  204m 

>6 

1 .94% 

None 

None 

None 

None 

Correct 

Correct 

ARA-1 

344m 

<6 

0.66% 

5 

73 

1 

31 

20.00% 

42.47% 

BGR-1 

483m  X  961m 

<6 

1 .20% 

No  target  areas  with  density  less  than  6  per  acre. 

Table 


7: 


Target  identification  and  UXO  containment  performance  of  PNNL/Sandia  for 
the  eonservative  designs  separated  by  target  density 
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Type  of 
Performance 
Objective 

Primary 

Performance 

Criteria 

Expected 

Performance 

Performance 

Confirmation 

Method 

Actual 

Performance 

Quantitative 

%ofUXO 
encompassed  with 
suspected  bounded 
areas 

100%  unless 
in  completely 
inaccessible 

areas 

Compared  to 
actual 
simulated 
targets 

See  Table  17 

%  of  Target  areas 
identified  within 
suspected  bounded 
areas 

100%  unless 
in  completely 
inaccessible 

areas 

Compared  to 
actual 
simulated 
targets 

See  Table  17 

%ofUXO 
encompassed  within 
suspected  bounded 
areas  if  target  density 
is  greater  than  6 
anomalies  per  acre 
above  background. 

100%  unless 
in  completely 
inaccessible 

areas 

Compared  to 
actual 
simulated 
targets 

100% 

%  of  Target  areas 
identified  with 
suspected  bounded 
areas  if  target  density 
is  greater  than  6 
anomalies  per  acre 
above  background. 

100%  unless 
in  completely 
inaccessible 

areas 

Compared  to 
actual 
simulated 
targets 

100% 

Accurate  distinction 
between  target  areas 
and  high  density 
non-OE  related 

areas. 

No  non-target 
areas  flagged 
as  target  areas 

Compared  to 
actual 
simulated 
targets 

100% 

%  of  land  area 
covered  by  transects 

<10% 

Designed 

0.6%  -  1.94% 

Accurate  estimate  of 
anomaly  density  for 
each  suspected  target 
area 

Within  20% 

Compared  to 
actual 
simulated 
targets 

Non-BGR 
sites:  7.6% 
BGR:  21% 

Qualitative 

Minimal  amount  of 
area  outside  of  true 
target  areas  included 
in  bounded  suspected 
target  areas. 

Minimize 

Compared  to 
actual 
simulated 
targets 

Minimized 

Table  18:  Performance  objective  results  for  the  PNNL/Sandia  team 
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5.2  Issues  Associated  with  Simulated  Sites  Anomaly  Distributions 

During  the  course  of  this  exercise,  a  number  of  inconsistencies  between  the  transect  data 
and  the  conceptual  site  models  were  noticed.  At  the  conclusion  of  the  exercise,  when  all 
of  the  simulated  data  were  provided  to  the  PNNL/Sandia  team  by  Mitretek  for  a  “post¬ 
mortem”  analysis,  the  causes  of  these  inconsistencies  and  their  full  impact  on  the 
characterization  process  and  conclusions  drawn  from  the  characterization  data  were  made 
clear.  The  significant  inconsistencies  in  the  simulated  data  are  highlighted  below  and 
then  examined  in  detail  in  Appendix  B  at  the  end  of  this  report. 

•  The  simulated  4.2”  mortar  target  areas  seemed  to  be  much  larger  than  standard 
4.2”  mortar  target  areas. 

•  The  simulated  bombing  range  target  areas  in  BGR-1  were  much  smaller  than  the 
assumed  munitions  would  indicate. 

•  The  dispersion  of  the  4.2”  mortar  target  areas  seem  to  have  a  peculiar  “cross” 
pattern  where  the  anomaly  density  is  highest  along  the  north-south  and  east-west 
axes  of  the  target  area. 

•  The  BGR-1  site  had  target  areas  with  a  high  percentage  of  UXO  in  relation  to 
ordnance  fragments  (27%)  in  each  target  area 

•  The  UXO  in  the  BGR-1  site  was  uniformly  spread  throughout  the  entire  target 
area.  Under  standard  assumptions  the  UXO  is  contained  in  the  area  where  the 
munitions  would  land  (Inside  the  range  and  deflection  probable  errors— Rpe  and 
Dpe.)  not  among  the  outer  locations  of  the  fragments. 

•  The  Ft.  Ritchie  “real”  data  as  applied  in  this  simulation  did  not  provide 
information  about  the  location  of  non-surveyed  parking  lots  and  buildings. 
Transects  that  covered  parking  lots  and  buildings  were  reported  to  the 
PNNL/Sandia  team  as  no  anomalies  found  instead  of  not  able  to  perform  desired 
transects. 

•  The  background  level  for  the  western  area  of  ARA-2B  was  2  anomalies  per  acre 
and  did  not  follow  the  assumed  uniform  distribution  of  background  used  through¬ 
out  this  simulation. 
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Appendix  A 


Variogram  Parameters 
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This  Appendix  contains  an  explanation  and  listing  of  the  various  parameters  used  in 
modeling  the  variograms  for  each  of  the  range  scenarios  for  which  kriging  estimates  were 
developed. 

Variograms  display  how  the  variance  between  data  points  separated  by  a  particular 
distance  changes  as  that  distance  increases.  Typically  this  separation  based  variance 
increases  as  the  separation  distance,  or  lag,  increases.  Figure  lA  shows  typical 
parameters  used  in  modeling  variograms. 

Commonly,  the  variance  values  in  a  variogram  level  off  after  reaching  some  separation 
distance.  The  distance  at  which  the  variance  tends  to  level  off  is  referred  to  as  the  range 
of  the  variogram  model.  The  variance  at  which  the  variogram  levels  off  is  termed  the  sill. 
In  some  cases,  the  experimental  variogram  data  will  display  some  variance  even  at  a  zero 
separation  distance.  If  this  is  the  case,  then  this  is  modeled  by  an  initial  variance  value 
referred  to  as  the  nugget. 

Separate  variogram  models  were  developed  for  each  of  the  kriging  estimate  scenarios 
presented  in  the  main  text.  The  parameters  of  these  models  are  presented  in  Tables  A1 
and  A2. 


Figure  A1 :  Example  variogram  showing  parameters  used  for  model  specification. 
Example  shown  is  indicator  variogram  using  a  threshold  of  65  anomalies  per  acre  for 
ARA-1.  Points  represent  values  computed  from  transect  data;  solid  lines  represent 
models  fit  to  data  points. 
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Site 

ARA-1 

ARA-2A 

ARA-2A 

ARA-2B 

ARA-2B 

Transect 
Spacing  (m) 
344 

298 

344 

345 

282 

Model  Type 

Ex 

Ex 

Sp 

Ex 

Sp 

Ex 

Sp 

Nugget 

0 

0 

0 

0 

0 

0 

0 

Sill 

0.015 

0.026 

0.017 

0.008 

0.034 

0.008 

0.034 

Range  (m) 

610 

1200 

920 

500 

2500 

500 

2500 

SRA-1 

438 

Ex 

0 

0.0445 

215 

SRA-1 

204 

Ex 

0 

0.037 

250 

BGR-1 

230 

Sp 

0 

0.028 

900 

BGR-1 

961/483 

Sp 

0 

0.03 

890 

Table  A1 :  Variogram  model  parameters  used  for  Indicator  Kriging  for  each  range 
scenario.  Model  types  are  given  as:  Ex=exponential  and  Sp=Spherical.  Multiple  lines  per 
scenario  indicate  a  compound  model  was  used.  All  models  were  isotropic. 


Site 

Transect 
Spacing  (m) 

Model  Type 

Nugget 

Sill 

Range (m) 

ARA-1 

344 

Sp 

0 

120 

430 

Ex 

0 

100 

4000 

ARA-1 

(6  transects) 

Ex 

0 

127 

900 

ARA-1 

(residuals) 

(6  transects) 

Sp 

0 

470 

2300 

ARA-2A 

298 

Sp 

0 

350 

1000 

ARA-2A 

344 

Sp 

0 

235 

900 

ARA-2B 

345 

Ga 

0 

95 

800 

Sp 

0 

240 

2500 

ARA-2B 

282 

Ga 

0 

95 

800 

Sp 

0 

240 

2500 

SRA-1 

438 

Ex 

0 

215 

235 

SRA-1 

204 

Ex 

0 

210 

340 

BGR-1 

230 

Sp 

0 

710 

1250 

BGR-1 

961/483 

Sp 

0 

760 

1400 

Table  A2:  Variogram  model  parameters  used  for  anomaly  density  Kriging  for  each  range 
scenario.  Model  types  are  given  as:  Ex=exponential,  Ga=Gaussian,  and  Sp=Spherical. 
Multiple  lines  per  scenario  indicate  a  compound  model  was  used.  All  models  were 
isotropic. 
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Appendix  B 


Issues  Associated  with  Simulated  Sites  Anomaiy  Distributions 
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Following  the  final  target  delineation,  PNNL  and  SNL  were  provided  with  the 
magnetic  anomaly  data  that  were  used  to  represent  the  simulated  sites.  These  data  were 
provided  to  the  site  characterization  team  to  be  used  in  a  very  detailed  assessment  of  the 
results  of  the  site  characterization  activities.  Typically  these  data  consisted  of  X  and  Y 
coordinate  values  for  each  simulated  anomaly.  The  simulated  anomalies  were  typically 
grouped  as  general  background  clutter,  munitions  fragments,  or  actual  UXO. 

Upon  review  of  these  data,  various  irregularities  were  noticed  in  the  anomaly  data.  Here, 
irregularities  refer  to  conditions  where  the  simulated  anomalies  are  significantly  different 
from  what  would  typically  be  expected  for  the  type  of  site  or  process  being  simulated. 

For  simulated  ranges  ARA-1,  ARA-2A,  and  ARA-2B  irregularities  were  seen  in  the 
distribution  of  munitions  fragments  for  each  of  the  target  areas.  Figures  B1  through  B3 
show  examples  of  the  patterns  of  munitions  fragments  observed  for  these  ranges.  Most 
notable  from  these  examples  are  the  rectangular  footprint  described  by  the  anomaly 
distribution  for  each  target,  and  the  cross-shaped  pattern  created  by  higher  density  areas 
oriented  in  a  north-south,  east- west  configuration  centered  on  each  individual  target. 

These  patterns  are  unexpected  for  explosive  fragmentation  and  are  not  explained  by  the 
firing  point  to  target  alignment  as  all  targets  have  the  north-south  and  east-west 
orientations  regardless  of  the  relative  orientation  between  the  target  and  the  firing 
location.  This  type  of  pattern  was  not  observed  in  the  BGR- 1  anomaly  data.  The  impact 
of  this  anomaly  pattern  on  the  transect  sampling  results  is  not  clear.  The  cross  pattern 
increases  the  anomaly  density  along  the  central  axes  of  the  target,  but  at  the  cost  of 
decreasing  density  everywhere  else.  This  would  tend  to  increase  potential  recognition  of 
the  site  as  an  area  with  anomaly  densities  above  the  general  clutter,  but  may  have  a 
negative  impact  on  recognizing  the  area  as  a  target  because  the  area  of  higher  densities  is 
reduced  from  what  would  be  expected.  A  transect  crossing  one  arm  of  the  cross  pattern 
might  show  high  density  results  only  at  the  arm  itself,  which  may  result  in  the  target  not 
being  recognized.  Conversely,  the  higher-density  arms  my  aid  in  the  recognition  of  very 
low  density  targets.  Determining  the  actual  impact  of  this  irregularity  requires  analysis  of 
the  contrast  between  the  target  and  background  clutter  densities  on  a  case-by-case  basis. 

In  all  cases,  the  unusual  anomaly  patterns  for  these  targets  would  have  a  direct  impact  on 
the  shape  of  the  target  boundaries  delineated  from  the  transect  data. 

For  simulated  range  ARA-2A,  data  from  a  field  investigation  conducted  at  Fort  Ritchie 
were  included  in  the  magnetic  anomaly  data  set.  Figure  B4  shows  the  distribution  of 
these  data.  There  are  two  irregularities  associated  with  this  data  set.  First  is  that  the 
distribution  of  anomalies  in  the  eastern  half  of  the  data  set  contains  many  apparent  blank 
areas.  These  are  apparently  due  to  areas  at  Fort  Ritchie  that  were  not  investigated 
because  they  contained  buildings  or  parking  lots.  Unfortunately,  this  information  is  not 
reflected  in  the  transect  sampling  results.  Instead  of  being  classified  as  un-sampled  areas, 
these  areas  appear  as  areas  which  were  sampled,  if  crossed  by  a  transect,  but  which 
contain  no  anomalies.  Transects  provided  to  the  site  characterization  team  in  these  areas 
reported  zero  anomalies  and  made  detection  of  the  targets  and  delineation  of  target 
boundaries  more  difficult  than  if  these  data  were  correctly  classified  as  being  impossible 
to  survey.  The  second  irregularity  for  this  data  set  is  in  the  western  half  of  the  data  where 
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there  is  a  large  coneentration  of  UXO.  This  area  is  eomprised  entirely  of  UXO  and 
contains  no  fragment  anomalies.  It  is  unexpected  to  have  a  target  area  with  100%  UXO 
and  no  other  site-related  anomalies.  The  lack  of  fragment  anomalies  made  detection  of 
this  area  challenging  due  to  the  low  anomaly  density. 

The  background  clutter  distribution  for  ARA-2A  appears  inconsistent  with  the  simulated 
site  design.  The  background  clutter  is  distributed  over  a  much  larger  area  than  the  range 
boundary  itself  This  leads  to  an  overall  site  clutter  density  lower  than  listed  as  the 
design  specification,  and  imparts  a  northeast-to-southwest  gradient  in  the  clutter  density 
across  the  range.  Figure  B5  shows  the  distribution  of  the  clutter  anomalies  for  ARA-2A. 
The  design  specification  for  overall  site  clutter  density  is  25  anomalies  per  hectare;  the 
actual  clutter  anomaly  density  for  the  area  within  the  range  boundary  is  1 8  anomalies  per 
hectare.  This  lower  background  density  acted  to  make  target  identification  from  transects 
within  this  area  easier  than  originally  planned. 

In  a  similar  nature,  the  clutter  distribution  for  the  ARA-2B  site  (Figure  B6)  also  showed 
some  irregularities.  The  clutter  anomalies  for  the  western  1/3  of  the  site  appear  to  be 
distributed  over  an  area  falling  well  outside  of  the  range  boundary.  The  intended 
anomaly  density  for  this  region  is  10  anomalies  per  hectare;  the  measured  density  within 
the  range  boundary  is  5.5  anomalies  per  hectare.  This  is  a  much  larger  contrast  in  clutter 
density  when  compared  against  the  15  anomaly  per  hectare  density  of  the  eastern  2/3 ’s  of 
the  site.  This  large  change  in  clutter  density  may  have  contributed  to  missing  the  target 
areas  in  this  region  during  the  characterization  team’s  target  delineation. 

The  target  areas  for  the  BGR-1  range  consisted  of  a  series  of  high  anomaly  density  areas 
scattered  across  the  site.  Typically  these  were  circular  in  nature  with  radii  varying  from 
100  m  to  600  m.  Figure  B7  shows  a  comparison  between  representative  target  areas  from 
ARA-1  and  ARA-2A  which  were  designated  as  mortar  and  Ml 55  practice  ranges,  and 
various  BGR-1  targets  which  were  designated  as  bombing  practice  targets.  Of  note  in 
this  figure  is  the  relative  size  difference  between  the  BGR-1  bombing  targets  and  the 
other  targets.  Many  of  the  BGR-1  targets  areas  are  considerably  smaller  than  the 
example  mortar  and  M155  targets.  The  small  size  of  some  of  the  BGR-1  targets  is 
unexpected  for  an  aerial  bombing  target,  particularly  when  compared  to  the  size  of  the 
mortar  and  Ml 55  targets  from  the  other  ranges  in  the  simulation. 

The  transect  designs  in  the  4.2”  mortar  sites  were  much  tighter  than  necessary  to  attain 
the  desired  probability  of  traversal,  but  the  tighter  transects  gave  the  planned  transect 
design  greater  ability  to  detect  the  lower  density  mortar  targets  simulated  by  Mitretek. 
Likewise,  many  of  the  bombing  and  gunnery  range  target  areas  were  different  from  the 
assumed  target  area  size  and  shape.  In  the  BGR-1  range  the  simulated  target  areas  were 
smaller  than  the  assumed  target  area  size  for  the  original  transect  designs  which  could 
have  led  to  many  unidentified  target  areas  in  the  BGR-1  site.  However,  a  few  of  the 
smaller  target  areas  were  traversed  and  identified  by  transects  in  the  original  sparse 
designs  which  lead  the  site  characterization  team  to  request  supplemental  transects  over 
the  entire  BGR-1  site  based  on  smaller  target  area  size  assumptions. 
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Figure  B8  shows  the  distribution  of  UXO  and  fragment  clutter  for  Target  1  of  the  BGR-1 
range.  As  seen  in  this  figure,  a  very  high  fraction  of  the  anomalies  are  characterized  as 
UXO,  and  these  UXO  are  distributed  more  uniformly  across  the  site  than  might  be 
expected.  In  addition,  the  anomaly  density  along  the  outer  edge  of  the  target  area  is  more 
uniformly  distributed,  which  creates  a  distinct  line  of  demarcation  around  the  target 
anomalies,  than  might  be  expected.  The  fragmentation  pattern  of  the  outer  edge  from  an 
actual  target  area  may  have  a  more  gradational  boundary  that  would  make  it  more 
difficult  to  identify  the  edges  of  the  target  areas. 

The  irregularities  observed  in  the  anomaly  distribution  of  the  simulated  ranges  may,  in 
some  cases,  have  had  a  direct  impact  on  the  ability  of  the  Labs  to  successfully  delineate 
the  simulated  impact  areas.  Whether  this  impact  would  have  resulted  in  greater  or  lesser 
success  can  only  be  determined  through  further  investigation  on  a  case-by-case  basis  and 
is  beyond  the  scope  of  this  report.  The  primary  purpose  of  this  appendix  is  to  document 
these  irregularities  so  that  the  results  of  the  target  identification  analyses  can  be  viewed  in 
a  proper  context. 
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Figure  B 1 :  Spatial  distribution  of  munitions  fragments  and  UXO  for  range  ARA- 1 . 
General  background  clutter  not  shown. 
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Figure  B2:  Spatial  distribution  of  munitions  fragments  and  UXO  for  range  ARA-2A. 

General  background  clutter  not  shown. 
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Figure  B3:  Spatial  distribution  of  munitions  fragments  and  UXO  for  southern  target 
eluster  of  range  ARA-2A.  General  baekground  elutter  not  shown. 
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Figure  B4:  Distribution  of  anomalies  for  the  Fort  Ritehie  data  set  as  included  in  the 
ARA-2A  simulated  range. 
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Figure  B5:  Background  clutter  density  for  range  ARA-2A. 


103 


24000  28000  32000  36000  40000 


16000  19000  22000  25000  28000  31000  34000  37000  40000  43000  46000 


50000  53000 


59000 

a 


68000  71000 


74000 

_l_ 


□  Range  Boundary 
I  I  Simulated  Impact  Areas 
•  ARA2B  Clutter 


I  Kilometers 


12 


16 


1  r 

50000  53000 


83000  86000 

_l_ 


n  r 

83000  86000 


Figure  B6: 


Background  clutter  density  for  range  ARA-2B. 
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Figure  B7:  Comparison  of  different  target  sizes  for  ARA-2A,  ARA-1,  and  BGR-1 
ranges. 
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Figure  B8:  Distribution  of  UXO  and  fragment  clutter  for  Target  Area  1  of  the  BGR-1 

range. 
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